2022-06-22 13:56:29,884 INFO MainThread:384207 [internal.py:wandb_internal():90] W&B internal server running at pid: 384207, started at: 2022-06-22 13:56:29.884594 2022-06-22 13:56:29,887 INFO WriterThread:384207 [datastore.py:open_for_write():75] open: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/run-3qxrqni1.wandb 2022-06-22 13:56:29,890 DEBUG SenderThread:384207 [sender.py:send():232] send: header 2022-06-22 13:56:29,890 DEBUG SenderThread:384207 [sender.py:send():232] send: run 2022-06-22 13:56:30,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: check_version 2022-06-22 13:56:30,178 INFO SenderThread:384207 [dir_watcher.py:__init__():166] watching files in: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files 2022-06-22 13:56:30,178 INFO SenderThread:384207 [sender.py:_start_run_threads():811] run started: 3qxrqni1 with start time 1655898989 2022-06-22 13:56:30,178 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 13:56:30,178 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 13:56:30,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: check_version 2022-06-22 13:56:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: run_start 2022-06-22 13:56:30,340 DEBUG HandlerThread:384207 [meta.py:__init__():35] meta init 2022-06-22 13:56:30,340 DEBUG HandlerThread:384207 [meta.py:__init__():49] meta init done 2022-06-22 13:56:30,340 DEBUG HandlerThread:384207 [meta.py:probe():209] probe 2022-06-22 13:56:30,341 DEBUG HandlerThread:384207 [meta.py:_setup_git():199] setup git 2022-06-22 13:56:30,358 DEBUG HandlerThread:384207 [meta.py:_setup_git():206] setup git done 2022-06-22 13:56:30,358 DEBUG HandlerThread:384207 [meta.py:_save_pip():53] save pip 2022-06-22 13:56:30,358 DEBUG HandlerThread:384207 [meta.py:_save_pip():67] save pip done 2022-06-22 13:56:30,358 DEBUG HandlerThread:384207 [meta.py:probe():247] probe done 2022-06-22 13:56:30,363 DEBUG SenderThread:384207 [sender.py:send():232] send: files 2022-06-22 13:56:30,364 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-metadata.json with policy now 2022-06-22 13:56:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:56:30,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:56:30,680 DEBUG SenderThread:384207 [sender.py:send():232] send: telemetry 2022-06-22 13:56:30,680 DEBUG SenderThread:384207 [sender.py:send():232] send: config 2022-06-22 13:56:30,681 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 13:56:30,681 DEBUG SenderThread:384207 [sender.py:send():232] send: telemetry 2022-06-22 13:56:30,681 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 13:56:30,681 WARNING SenderThread:384207 [sender.py:send_metric():904] Seen metric with glob (shouldn't happen) 2022-06-22 13:56:30,682 DEBUG SenderThread:384207 [sender.py:send():232] send: telemetry 2022-06-22 13:56:31,083 INFO Thread-11 :384207 [upload_job.py:push():137] Uploaded file /tmp/tmppesvi0w0wandb/hz3pjwbz-wandb-metadata.json 2022-06-22 13:56:31,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_created():213] file/dir created: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 13:56:31,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_created():213] file/dir created: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:56:31,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_created():213] file/dir created: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-metadata.json 2022-06-22 13:56:31,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_created():213] file/dir created: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/requirements.txt 2022-06-22 13:56:33,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:56:45,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:56:45,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:56:58,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:57:01,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/config.yaml 2022-06-22 13:57:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:57:01,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:57:16,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:57:16,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:57:28,483 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:57:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:57:32,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:57:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:57:47,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:57:58,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:58:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:58:03,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:58:18,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:58:18,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:58:28,634 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:58:33,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:58:33,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:58:48,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:58:48,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:58:58,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:59:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:59:04,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:59:12,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:59:19,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:59:27,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:28,779 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:59:29,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:31,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:33,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:34,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:59:34,866 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:59:35,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:37,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:39,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:41,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:43,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:45,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:47,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:49,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:50,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 13:59:50,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 13:59:51,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:53,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:55,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:57,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 13:59:58,853 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 13:59:59,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:05,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:00:05,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:00:09,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:15,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:19,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:20,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:00:20,599 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:00:23,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:27,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:28,931 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:00:29,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:33,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:35,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:00:35,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:00:37,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:39,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:43,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:45,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:49,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:51,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:00:51,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:00:53,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:55,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:57,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:00:59,003 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:00:59,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:03,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:03,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:04,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:05,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:01:06,456 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:01:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:06,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:07,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:07,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:08,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:09,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:10,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:12,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:01:15,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:17,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:20,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:21,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:01:21,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:01:22,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:24,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:26,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:28,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:29,080 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:01:30,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:32,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:34,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:36,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:01:36,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:01:38,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:40,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:42,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:44,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:46,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:48,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:50,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:52,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:01:52,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:01:52,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:54,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:56,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:01:59,154 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:02:06,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:07,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:02:07,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:02:10,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:16,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:18,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:22,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:22,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:02:22,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:02:26,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:29,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:02:30,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:32,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:36,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:38,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:02:38,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:02:38,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:40,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:44,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:46,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:48,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:52,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:53,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:02:53,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:02:54,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:56,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:56,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:57,744 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,749 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,755 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,760 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,766 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,771 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,781 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,786 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,792 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,797 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,803 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,808 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,808 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,814 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,819 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,825 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,830 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,836 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,840 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,840 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,841 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,841 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,841 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,841 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,841 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,841 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,843 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,844 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,845 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,846 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,847 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,848 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,849 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,850 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,851 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,853 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,855 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,856 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,857 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,858 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,859 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,859 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,859 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,859 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,859 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,859 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,862 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,863 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,864 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,865 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,866 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,867 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,868 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,869 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,870 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,871 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,873 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,874 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,875 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,876 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,877 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,878 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,879 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,880 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,881 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,882 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,883 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,884 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,885 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,886 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,887 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,888 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,889 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,890 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,892 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,893 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,894 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,895 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,896 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,897 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,898 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,899 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,900 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,901 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,902 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,903 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,904 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,905 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,906 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,907 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,908 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,909 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,910 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,911 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,912 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,913 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,914 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,916 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,917 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,919 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,920 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,921 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,922 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,923 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,924 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,925 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,926 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,927 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,928 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,929 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,930 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,931 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,932 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,933 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,934 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,935 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,936 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,937 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,938 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,939 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,940 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,941 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,942 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,943 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,944 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,946 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,947 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,948 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,949 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,950 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,952 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,953 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 14:02:57,954 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 14:02:58,010 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 14:02:58,073 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 14:02:58,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 14:02:58,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:02:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:58,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:02:59,321 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:02:59,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:00,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:00,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:03,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:05,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:05,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:07,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/config.yaml 2022-06-22 14:03:07,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:07,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:03:08,579 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:03:08,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:08,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:11,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:11,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:12,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:13,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:03:13,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:15,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:17,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:19,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:21,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:23,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:03:23,807 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:03:25,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:27,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:29,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:29,417 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:03:31,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:33,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:35,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:37,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:39,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:03:39,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:03:39,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:41,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:43,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:45,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:47,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:49,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:54,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:03:54,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:03:57,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:03:59,494 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:04:01,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:05,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:09,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:04:09,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:04:13,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:17,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:21,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:04:24,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:04:25,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:27,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:29,569 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:04:31,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:33,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:37,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:39,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:04:40,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:04:43,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:45,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:47,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:51,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:51,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:53,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:54,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:04:55,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:04:55,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:55,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:57,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:58,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:04:59,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:04:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:04:59,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:00,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:01,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:01,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:02,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:03,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:05,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:06,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:07,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:08,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:09,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:10,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:10,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:05:10,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:05:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:11,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:12,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:05:13,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:15,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:17,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:19,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:21,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:24,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:26,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:05:26,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:05:26,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:28,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:29,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:05:30,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:32,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:34,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:36,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:38,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:40,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:41,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:05:41,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:05:42,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:44,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:54,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:56,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:05:56,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:05:58,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:05:59,799 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:06:02,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:06,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:10,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:06:11,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:06:14,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:18,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:20,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:24,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:26,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:06:26,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:06:29,874 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:06:30,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:32,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:36,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:38,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:40,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:06:42,376 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:06:44,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:46,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:46,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:46,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:47,082 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 14:06:47,285 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 14:06:47,345 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 14:06:47,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 14:06:48,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:50,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:52,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:54,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:55,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:55,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:57,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:06:57,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:06:58,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:58,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:06:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:06:59,980 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:07:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:00,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:01,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:01,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:02,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:03,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:03,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:04,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:05,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:06,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:07,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:08,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:09,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:09,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:10,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:11,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:07:12,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:07:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:13,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:07:14,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:16,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:18,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:20,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:22,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:24,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:26,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:28,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:07:28,107 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:07:28,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:30,062 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:07:30,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:32,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:34,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:36,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:38,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:07:43,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:07:46,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:50,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:54,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:58,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:07:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:07:58,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:08:00,139 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:08:02,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:06,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:08,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:12,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:13,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:08:13,788 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:08:16,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:18,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:23,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:25,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:27,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:08:29,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:08:30,218 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:08:31,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:33,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:35,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:38,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:38,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:39,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:41,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:43,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:08:44,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:08:45,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:47,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:49,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:50,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:51,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:52,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:53,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:54,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:55,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:55,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:56,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:57,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:57,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:08:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:08:59,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:08:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:08:59,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,295 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:09:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:00,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:01,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:02,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:03,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:04,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:05,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:06,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:09:07,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:09,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:11,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:13,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:14,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:09:14,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:09:15,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:17,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:19,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:21,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:23,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:25,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:27,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:29,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:09:29,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:09:30,371 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:09:39,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:43,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:45,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:09:45,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:09:47,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:51,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:55,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:09:59,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:10:00,440 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:10:00,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:10:01,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:05,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:09,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:11,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:15,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:15,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:10:15,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:10:17,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:19,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:23,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:25,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:27,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:29,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:10:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:30,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:31,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:31,116 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 14:10:31,285 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 14:10:31,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:10:31,347 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 14:10:31,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:10:31,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 14:10:33,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:35,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:37,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:39,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:41,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:43,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:45,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:46,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:10:46,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:10:47,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:48,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:48,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:49,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:49,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:50,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:51,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:52,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:53,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:55,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:55,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:57,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:58,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:10:59,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:10:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:00,622 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:11:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:01,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:11:01,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:11:02,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:02,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:03,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:03,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:04,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:04,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:11:06,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:08,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:10,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:12,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:14,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:16,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:17,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:11:17,125 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:11:18,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:20,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:22,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:30,697 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:11:32,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:11:32,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:11:32,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:36,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:42,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:46,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:11:47,785 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:11:50,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:52,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:11:56,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:00,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:00,771 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:12:02,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:03,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:12:03,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:12:06,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:08,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:12,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:14,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:18,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:18,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:12:18,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:12:20,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:22,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:24,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:25,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:25,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:25,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:25,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:26,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:28,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:30,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:30,845 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:12:32,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:33,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:12:33,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:12:34,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:37,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:39,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:41,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:43,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:45,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:47,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:48,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:12:49,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:12:49,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:49,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:50,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:51,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:52,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:53,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:54,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:55,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:57,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:57,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:12:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:12:59,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:00,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:13:01,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:03,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:13:04,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:13:04,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:13:05,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:07,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:09,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:11,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:13,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:15,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:17,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:13:19,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:13:25,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:29,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:30,992 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:13:33,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:34,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:13:34,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:13:37,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:41,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:45,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:47,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:50,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:13:50,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:13:51,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:55,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:13:57,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:01,070 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:14:01,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:03,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:05,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:14:05,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:14:05,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:09,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:11,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:13,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:15,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:16,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:16,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:17,624 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 14:14:17,839 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 14:14:17,909 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 14:14:18,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 14:14:19,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:20,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:14:20,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:14:21,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:23,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:25,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:27,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:29,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:31,171 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:14:31,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:33,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:35,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:14:36,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:14:37,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:40,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:42,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:43,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:43,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:44,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:45,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:45,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:46,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:47,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:48,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:49,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:14:51,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:14:51,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:51,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:52,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:52,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:53,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:54,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:55,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:55,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:56,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:14:57,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:14:58,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:00,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:00,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:00,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:15:01,245 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:15:02,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:04,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:06,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:15:06,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:15:06,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:08,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:16,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:21,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:15:21,795 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:15:22,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:26,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:30,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:31,325 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:15:34,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:36,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:15:37,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:15:40,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:44,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:46,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:50,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:15:52,414 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:15:52,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:56,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:15:58,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:00,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:01,404 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:16:04,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:06,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:16:07,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:16:08,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:09,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:10,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:14,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:16,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:18,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:20,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:22,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:16:22,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:16:24,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:26,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:28,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:30,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:31,483 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:16:32,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:34,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:34,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:36,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:38,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:16:38,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:16:38,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:38,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:39,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:40,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:41,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:42,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:43,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:43,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:44,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:45,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:45,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:46,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:46,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:47,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:48,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:49,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:49,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:50,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:51,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:51,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:52,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:53,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:53,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:16:53,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:16:54,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:56,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:16:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:16:58,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:00,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:01,558 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:17:08,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:17:08,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:17:10,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:14,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:18,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:22,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:24,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:17:24,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:17:26,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:30,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:31,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:17:35,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:37,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:39,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:17:39,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:17:41,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:45,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:47,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:49,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:53,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:54,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:17:54,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:17:55,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:17:59,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:01,708 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:18:01,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:03,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:04,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 14:18:05,011 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 14:18:05,245 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 14:18:05,304 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 14:18:05,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 14:18:06,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:07,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:09,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:18:10,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:18:11,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:13,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:15,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:17,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:19,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:21,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:23,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:25,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:18:25,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:18:25,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:27,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:29,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:31,786 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:18:31,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:33,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:35,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:37,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:39,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:40,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:18:40,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:18:41,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:44,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:46,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:48,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:50,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:52,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:54,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:55,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:18:55,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:18:56,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:18:58,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:00,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:01,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:19:02,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:04,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:06,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:08,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:10,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:11,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:19:11,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:19:12,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:14,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:16,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:18,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:20,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:22,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:24,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:26,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:19:26,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:19:26,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:28,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:30,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:31,933 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:19:32,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:34,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:36,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:38,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:40,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:41,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:19:41,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:19:42,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:44,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:46,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:48,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:50,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:52,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:54,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:56,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:19:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:19:57,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:19:58,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:01,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:02,010 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:20:03,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:05,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:07,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:09,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:11,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:12,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:20:12,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:20:13,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:15,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:17,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:19,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:21,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:23,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:25,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:27,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:20:27,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:20:27,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:29,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:31,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:32,086 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:20:33,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:35,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:37,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:39,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:41,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:20:43,063 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:20:43,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:45,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:47,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:49,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:51,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:53,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:55,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:57,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:20:58,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:20:58,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:20:59,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:01,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:02,161 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:21:03,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:05,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:07,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:09,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:11,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:13,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:15,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:21:15,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:21:15,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:18,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:20,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:22,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:24,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:26,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:28,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:30,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:21:30,801 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:21:30,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:32,232 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:21:32,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:34,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:36,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:38,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:40,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:42,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:44,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:46,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:21:46,057 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:21:46,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:48,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:50,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:52,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:54,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:56,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:21:58,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:00,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:01,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:22:01,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:22:02,304 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:22:02,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:04,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:06,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:08,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:10,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:12,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:14,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:22:16,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:22:16,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:18,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:20,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:22,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:24,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:26,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:28,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:30,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:31,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:22:31,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:22:32,377 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:22:32,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:35,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:37,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:39,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:41,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:43,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:45,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:47,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:22:47,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:22:47,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:49,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:51,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:53,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:55,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:57,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:22:59,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:01,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:02,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:23:02,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:23:02,666 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:23:03,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:05,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:07,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:09,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:11,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:13,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:15,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:17,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:23:17,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:23:17,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:19,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:21,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:23,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:25,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:27,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:29,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:31,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:32,525 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:23:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:23:33,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:23:33,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:35,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:37,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:39,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:41,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:43,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:45,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:47,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:48,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:23:48,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:23:49,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:51,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:53,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:55,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:57,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:23:59,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:01,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:02,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:24:03,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:24:03,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:24:03,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:05,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:08,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:10,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:12,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:14,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:16,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:18,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:24:19,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:24:20,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:22,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:24,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:26,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:28,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:30,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:32,681 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:24:32,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:34,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:24:34,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:24:34,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:36,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:38,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:40,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:42,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:44,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:46,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:48,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:49,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:24:49,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:24:50,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:52,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:54,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:56,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:24:58,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:00,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:02,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:25:02,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:04,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:25:05,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:25:06,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:08,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:10,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:12,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:14,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:16,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:18,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:20,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:25:20,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:25:20,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:22,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:24,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:26,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:28,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:30,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:32,833 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:25:32,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:34,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:35,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:25:35,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:25:36,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:38,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:40,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:42,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:44,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:46,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:48,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:50,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:25:50,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:25:50,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:52,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:54,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:56,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:25:59,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:02,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:02,910 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:26:04,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:06,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:06,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:26:06,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:26:08,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:10,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:12,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:14,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:16,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:18,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:20,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:21,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:26:21,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:26:22,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:24,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:26,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:28,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:30,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:32,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:32,984 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:26:34,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:36,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:26:36,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:26:38,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:40,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:42,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:44,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:46,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:48,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:50,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:51,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:26:51,918 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:26:52,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:54,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:56,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:26:58,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:00,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:02,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:03,058 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:27:04,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:06,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:07,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:27:07,223 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:27:08,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:10,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:12,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:14,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:16,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:18,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:20,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:22,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:22,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:27:22,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:27:25,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:27,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:29,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:31,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:33,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:33,133 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:27:35,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:37,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:37,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:27:37,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:27:39,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:41,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:43,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:45,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:47,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:49,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:51,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:53,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:53,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:27:53,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:27:55,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:57,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:27:59,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:01,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:03,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:03,211 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:28:05,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:07,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:08,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:28:08,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:28:09,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:11,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:13,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:15,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:17,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:19,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:21,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:23,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:23,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:28:23,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:28:25,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:28,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:30,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:32,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:33,287 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:28:34,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:36,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:38,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:38,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:28:38,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:28:40,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:42,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:44,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:46,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:48,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:50,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:52,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:54,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:54,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:28:54,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:28:56,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:28:58,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:00,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:02,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:03,362 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:29:04,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:06,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:08,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:29:09,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:29:10,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:12,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:14,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:16,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:18,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:20,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:22,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:24,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:29:24,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:29:26,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:28,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:30,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:33,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:33,436 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:29:35,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:37,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:39,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:29:39,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:29:41,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:43,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:45,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:47,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:49,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:51,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:53,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:55,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:55,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:29:55,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:29:57,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:29:59,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:01,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:03,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:03,511 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:30:05,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:07,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:09,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:10,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:30:10,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:30:11,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:13,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:15,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:17,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:19,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:21,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:23,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:25,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:25,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:30:25,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:30:27,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:29,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:31,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:33,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:33,587 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:30:35,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:37,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:39,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:41,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:30:41,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:30:42,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:44,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:46,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:48,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:50,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:52,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:54,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:56,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:30:56,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:30:56,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:30:58,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:00,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:02,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:03,665 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:31:04,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:06,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:08,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:10,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:31:11,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:31:12,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:14,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:16,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:18,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:20,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:22,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:24,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:26,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:26,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:31:26,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:31:28,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:30,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:32,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:33,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:31:34,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:36,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:38,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:40,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:42,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:31:42,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:31:42,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:45,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:47,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:49,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:51,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:53,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:55,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:57,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:31:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:31:57,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:31:59,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:01,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:03,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:03,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:32:05,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:07,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:09,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:11,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:12,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:32:12,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:32:13,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:15,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:17,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:19,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:21,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:23,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:25,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:27,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:28,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:32:28,359 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:32:29,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:31,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:33,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:33,887 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:32:35,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:37,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:39,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:41,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:43,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:43,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:32:43,597 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:32:45,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:47,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:49,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:51,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:53,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:55,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:57,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:32:58,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:32:58,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:32:59,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:01,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:03,964 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:33:04,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:06,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:08,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:10,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:12,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:14,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:14,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:33:14,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:33:16,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:18,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:20,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:22,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:24,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:26,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:28,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:33:29,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:33:30,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:32,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:34,039 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:33:34,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:36,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:38,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:40,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:42,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:44,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:44,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:33:44,713 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:33:46,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:48,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:50,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:52,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:54,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:56,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:33:58,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:34:00,058 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:34:00,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:02,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:04,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:34:04,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:06,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:08,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:10,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:12,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:14,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:15,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:34:15,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:34:16,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:18,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:21,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:23,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:25,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:27,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:29,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:30,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:34:30,560 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:34:31,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:33,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:34,187 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:34:35,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:37,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:39,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:41,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:43,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:45,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:45,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:34:45,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:34:47,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:49,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:51,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:53,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:55,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:57,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:34:59,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:01,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:35:01,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:35:01,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:03,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:04,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:35:05,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:07,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:09,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:11,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:13,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:15,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:16,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:35:16,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:35:17,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:19,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:21,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:23,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:25,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:27,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:29,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:31,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:31,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:35:31,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:35:33,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:34,337 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:35:35,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:37,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:39,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:41,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:44,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:46,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:46,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:35:46,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:35:48,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:50,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:52,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:54,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:56,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:35:58,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:00,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:36:02,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:36:02,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:04,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:04,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:36:06,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:08,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:10,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:12,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:14,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:16,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:17,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:36:17,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:36:18,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:20,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:22,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:24,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:26,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:28,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:30,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:32,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:36:32,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:36:34,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:34,488 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:36:36,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:38,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:40,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:42,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:44,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:46,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:36:48,012 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:36:48,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:50,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:52,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:54,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:56,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:36:58,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:00,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:02,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:03,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:37:03,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:37:04,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:04,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:37:06,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:08,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:10,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:13,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:15,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:17,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:18,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:37:18,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:37:19,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:21,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:23,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:25,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:27,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:29,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:31,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:33,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:33,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:37:33,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:37:34,637 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:37:35,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:37,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:39,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:41,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:43,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:45,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:47,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:49,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:37:49,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:37:49,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:51,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:53,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:55,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:57,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:37:59,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:01,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:03,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:38:04,422 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:38:04,710 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:38:05,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:07,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:09,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:11,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:14,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:16,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:18,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:38:19,720 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:38:20,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:22,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:24,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:26,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:28,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:30,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:32,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:34,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:34,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:38:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:38:35,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:38:36,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:38,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:40,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:42,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:44,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:46,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:48,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:50,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:38:50,254 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:38:50,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:52,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:54,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:56,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:38:58,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:00,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:02,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:04,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:04,856 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:39:05,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:39:05,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:39:06,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:08,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:10,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:12,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:14,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:16,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:18,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:20,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:20,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:39:20,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:39:22,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:24,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:26,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:28,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:30,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:32,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:34,926 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:39:35,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:39:36,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:39:37,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:39,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:41,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:43,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:45,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:47,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:49,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:51,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:39:51,557 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:39:53,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:55,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:57,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:39:59,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:01,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:03,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:05,002 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:40:05,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:06,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:40:06,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:40:07,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:09,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:11,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:13,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:15,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:17,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:19,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:21,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:40:22,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:40:23,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:25,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:27,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:29,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:31,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:33,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:35,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:40:35,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:37,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:40:37,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:40:37,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:39,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:41,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:43,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:45,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:47,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:49,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:51,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:52,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:40:52,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:40:53,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:55,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:57,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:40:59,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:01,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:03,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:05,148 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:41:05,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:07,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:41:07,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:41:09,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:11,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:13,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:15,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:17,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:19,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:21,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:41:23,127 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:41:23,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:25,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:27,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:29,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:31,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:34,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:35,220 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:41:36,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:38,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:41:38,359 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:41:38,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:40,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:42,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:44,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:46,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:48,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:50,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:52,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:53,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:41:53,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:41:54,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:56,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:41:58,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:00,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:02,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:04,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:05,297 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:42:06,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:08,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:42:08,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:42:10,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:12,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:14,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:16,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:18,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:20,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:22,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:24,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:42:24,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:42:24,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:26,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:28,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:30,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:32,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:34,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:35,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:42:36,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:38,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:42:39,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:42:40,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:42,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:44,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:46,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:48,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:50,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:52,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:54,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:42:54,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:42:55,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:57,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:42:59,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:01,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:03,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:05,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:05,445 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:43:07,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:09,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:43:09,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:43:11,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:13,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:15,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:17,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:19,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:21,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:23,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:25,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:43:25,138 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:43:25,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:27,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:29,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:31,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:33,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:35,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:35,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:43:37,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:39,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:43:40,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:43:41,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:43,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:45,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:47,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:49,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:51,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:53,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:55,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:55,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:43:55,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:43:57,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:43:59,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:01,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:03,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:05,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:05,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:44:07,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:09,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:10,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:44:10,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:44:11,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:13,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:15,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:17,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:19,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:21,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:23,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:26,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:44:26,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:44:26,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:28,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:30,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:32,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:34,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:35,662 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:44:36,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:38,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:40,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:44:41,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:44:42,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:44,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:46,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:48,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:50,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:52,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:54,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:56,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:44:56,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:44:56,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:44:58,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:00,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:02,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:04,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:05,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:45:06,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:08,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:10,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:11,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:45:11,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:45:12,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:14,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:16,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:18,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:20,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:22,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:24,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:26,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:45:27,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:45:28,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:30,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:32,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:34,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:35,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:45:36,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:38,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:40,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:45:42,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:45:42,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:44,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:46,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:48,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:50,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:52,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:55,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:57,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:45:57,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:45:57,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:45:59,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:01,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:03,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:05,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:05,896 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:46:07,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:09,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:11,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:13,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:46:13,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:46:13,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:15,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:17,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:19,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:21,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:23,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:25,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:27,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:46:28,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:46:29,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:31,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:33,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:35,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:35,970 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:46:37,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:39,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:41,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:43,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:45,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:46:45,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:46:48,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:50,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:52,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:54,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:56,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:46:58,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:00,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:00,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:47:00,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:47:02,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:04,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:06,047 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:47:06,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:08,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:10,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:12,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:14,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:47:16,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:47:16,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:18,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:20,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:22,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:24,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:26,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:28,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:30,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:31,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:47:31,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:47:32,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:34,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:36,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:47:36,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:38,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:40,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:42,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:44,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:46,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:46,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:47:46,807 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:47:48,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:51,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:53,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:55,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:57,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:47:59,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:01,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:02,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:48:02,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:48:03,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:05,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:06,197 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:48:07,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:09,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:11,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:13,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:15,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:17,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:48:17,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:48:17,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:19,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:21,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:23,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:25,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:27,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:29,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:31,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:32,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:48:32,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:48:33,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:35,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:36,273 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:48:37,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:39,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:41,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:43,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:45,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:47,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:48:47,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:48:49,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:51,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:53,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:55,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:57,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:48:59,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:01,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:03,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:49:03,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:49:03,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:05,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:06,350 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:49:07,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:09,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:11,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:13,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:16,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:18,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:49:18,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:49:18,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:20,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:22,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:24,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:26,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:28,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:30,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:32,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:49:33,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:49:34,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:36,423 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:49:36,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:38,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:40,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:42,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:44,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:46,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:48,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:48,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:49:48,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:49:50,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:52,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:54,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:56,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:49:58,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:00,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:02,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:04,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:50:04,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:50:04,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:06,500 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:50:06,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:08,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:10,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:12,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:14,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:16,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:18,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:50:19,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:50:20,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:22,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:24,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:26,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:28,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:30,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:32,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:34,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:50:34,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:50:36,573 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:50:36,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:38,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:40,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:42,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:44,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:46,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:48,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:49,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:50:49,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:50:50,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:52,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:54,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:56,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:50:58,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:00,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:02,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:04,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:05,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:51:05,183 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:51:06,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:06,642 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:51:08,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:10,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:12,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:14,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:16,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:18,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:20,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:51:20,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:51:22,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:24,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:27,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:29,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:31,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:33,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:35,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:51:35,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:51:36,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:51:37,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:39,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:41,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:43,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:45,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:47,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:49,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:51,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:51:51,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:51:51,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:53,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:55,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:57,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:51:59,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:01,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:03,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:05,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:06,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:52:06,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:52:06,793 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:52:07,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:09,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:11,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:13,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:15,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:17,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:19,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:21,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:52:21,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:52:23,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:25,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:27,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:29,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:31,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:33,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:35,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:36,870 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:52:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:52:37,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:52:37,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:39,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:42,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:44,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:46,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:48,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:50,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:52,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:52,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:52:52,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:52:54,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:56,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:52:58,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:00,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:02,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:04,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:06,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:06,942 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:53:07,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:53:07,963 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:53:08,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:10,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:12,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:14,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:16,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:18,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:20,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:22,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:23,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:53:23,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:53:24,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:26,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:28,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:30,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:32,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:34,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:36,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:37,018 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:53:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:53:38,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:53:38,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:40,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:42,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:44,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:46,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:48,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:50,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:52,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:53,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:53:53,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:53:54,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:56,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:53:58,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:00,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:02,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:04,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:06,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:07,096 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:54:08,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:09,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:54:09,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:54:10,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:13,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:15,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:17,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:19,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:21,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:23,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:24,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:54:24,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:54:25,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:27,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:29,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:31,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:33,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:35,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:37,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:54:37,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:39,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:39,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:54:39,969 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:54:41,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:43,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:45,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:47,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:49,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:51,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:53,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:55,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:54:55,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:54:55,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:57,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:54:59,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:01,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:03,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:05,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:07,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:55:07,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:09,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:55:10,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:55:11,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:13,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:15,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:17,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:19,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:21,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:23,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:25,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:55:25,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:55:27,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:29,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:31,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:33,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:35,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:37,324 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:55:37,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:39,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:41,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:55:41,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:55:42,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:44,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:46,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:48,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:50,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:52,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:54,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:56,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:55:56,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:55:56,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:55:58,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:00,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:02,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:04,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:06,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:07,400 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:56:08,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:10,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:11,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:56:11,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:56:12,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:14,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:16,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:18,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:20,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:22,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:24,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:26,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:56:26,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:56:28,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:30,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:32,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:34,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:36,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:37,474 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:56:38,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:40,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:42,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:56:42,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:56:42,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:44,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:46,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:48,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:50,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:52,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:54,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:56,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:56:57,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:56:57,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:56:58,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:00,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:02,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:04,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:06,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:07,551 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:57:08,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:11,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:12,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:57:12,869 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:57:13,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:15,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:17,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:19,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:21,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:23,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:25,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:27,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:57:28,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:57:29,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:31,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:33,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:35,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:37,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:57:37,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:39,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:41,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:43,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:57:43,311 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:57:43,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:45,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:47,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:49,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:51,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:53,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:55,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:57,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:57:58,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:57:58,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:57:59,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:01,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:03,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:05,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:07,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:58:07,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:09,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:11,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:13,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:13,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:58:13,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:58:15,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:18,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:20,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:22,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:24,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:26,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:28,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:29,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:58:29,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:58:30,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:32,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:34,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:36,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:37,786 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:58:38,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:40,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:42,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:44,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:58:44,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:58:44,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:46,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:48,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:50,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:52,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:54,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:56,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:58,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:58:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:58:59,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:59:00,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:02,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:04,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:06,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:07,860 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:59:08,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:10,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:12,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:14,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:59:14,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:59:16,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:18,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:20,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:22,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:24,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:26,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:28,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:59:30,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:59:30,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:32,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:34,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:36,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:37,934 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 14:59:38,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:40,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:42,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:44,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:45,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 14:59:45,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 14:59:46,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:48,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:50,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:52,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:54,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:56,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 14:59:58,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:00,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:00,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:00:00,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:00:02,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:04,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:06,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:08,012 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:00:08,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:11,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:13,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:15,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:16,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:00:16,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:00:17,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:19,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:21,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:23,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:25,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:27,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:29,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:31,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:00:31,457 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:00:31,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:33,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:35,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:37,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:38,088 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:00:39,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:41,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:43,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:45,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:46,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:00:46,687 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:00:47,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:49,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:51,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:53,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:55,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:57,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:00:59,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:01,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:01,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:01:01,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:01:03,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:05,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:07,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:08,171 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:01:09,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:11,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:13,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:15,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:01:17,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:01:17,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:19,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:22,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:24,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:26,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:28,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:30,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:01:32,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:01:32,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:01:38,246 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:01:47,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:01:47,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:02:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:02:03,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:02:08,321 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:02:18,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:02:18,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:02:28,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:02:28,194 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 15:02:28,199 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 15:02:28,205 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 15:02:28,210 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 15:02:28,216 DEBUG SenderThread:384207 [sender.py:send():232] send: metric 2022-06-22 15:02:28,221 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 15:02:28,247 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 15:02:28,323 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 15:02:28,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 15:02:29,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:02:30,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:02:33,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:02:33,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:02:34,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:02:38,404 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:02:39,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/config.yaml 2022-06-22 15:02:48,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:02:48,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:02:48,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:03:04,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:03:04,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:03:08,514 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:03:19,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:03:19,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:03:34,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:03:34,637 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:03:38,589 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:03:49,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:03:49,863 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:04:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:04:05,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:04:08,666 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:04:20,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:04:20,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:04:35,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:04:35,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:04:38,740 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:04:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:04:50,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:05:06,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:05:06,006 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:05:08,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:05:21,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:05:21,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:05:36,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:05:36,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:05:38,893 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:05:51,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:05:51,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:06:06,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:06:06,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:06:08,972 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:06:22,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:06:22,149 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:06:37,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:06:37,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:06:39,049 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:06:52,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:06:52,599 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:06:53,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:06:55,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:06:57,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:06:59,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:01,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:03,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:07:07,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:07:08,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:09,124 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:07:10,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:12,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:14,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:16,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:18,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:18,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:20,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:22,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:07:23,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:07:23,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:23,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:24,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:24,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:26,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:27,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:28,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:28,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:29,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:30,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:31,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:33,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:34,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:34,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:35,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:36,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:36,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:38,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:38,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:07:38,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:07:39,199 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:07:40,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:41,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:41,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:07:42,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:44,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:52,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:07:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:07:53,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:07:58,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:02,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:06,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:08:09,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:08:09,276 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:08:10,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:14,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:16,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:20,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:22,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:24,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:08:24,258 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:08:26,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:28,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:32,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:34,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:38,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:39,352 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:08:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:08:39,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:08:40,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:42,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:45,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:45,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:45,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:45,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:46,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:46,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:08:48,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:50,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:52,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:54,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:08:54,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:08:54,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:08:58,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:00,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:02,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:04,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:06,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:09,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:09,428 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:09:09,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:09:09,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:09:11,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:13,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:15,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:17,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:19,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:20,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:21,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:22,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:24,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:09:25,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:09:25,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:26,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:27,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:28,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:30,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:31,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:31,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:32,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:33,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:35,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:37,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:39,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:39,504 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:09:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:09:40,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:09:49,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:52,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:09:55,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:09:55,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:09:55,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:09:59,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:03,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:07,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:09,580 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:10:10,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:10:10,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:10:11,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:15,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:19,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:21,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:25,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:26,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:10:26,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:10:27,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:31,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:33,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:35,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:39,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:39,656 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:10:41,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:10:41,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:10:43,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:43,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:10:44,137 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 15:10:44,368 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 15:10:44,452 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 15:10:45,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 15:10:45,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:47,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:51,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:53,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:55,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:56,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:10:56,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:10:57,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:10:59,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:01,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:03,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:05,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:07,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:09,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:09,812 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:11:11,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:12,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:11:12,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:11:13,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:15,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:17,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:17,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:17,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:19,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:21,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:21,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:21,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:22,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:23,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:25,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:25,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:26,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:27,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:11:27,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:11:27,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:28,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:29,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:31,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:32,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:33,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:35,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:39,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:11:41,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:42,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:11:42,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:11:47,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:51,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:11:51,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:55,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:11:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:11:57,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:11:59,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:01,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:05,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:09,963 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:12:10,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:12,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:13,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:12:13,066 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:12:16,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:18,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:22,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:24,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:26,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:12:28,293 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:12:30,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:32,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:34,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:12:38,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:40,035 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:12:40,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:42,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:43,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:12:43,637 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:12:44,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:46,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:48,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:50,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:52,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:54,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:56,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:58,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:12:58,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:12:58,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:13:00,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:02,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:04,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:06,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:08,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:10,111 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:13:10,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:10,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:10,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:12,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:13:14,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:13:14,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:14,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:16,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:18,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:18,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:20,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:20,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:21,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:22,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:22,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:23,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:25,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:26,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:28,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:28,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:13:29,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:13:29,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:32,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:36,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:40,187 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:13:40,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:13:44,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:13:46,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:49,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:13:50,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:54,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:56,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:13:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:13:59,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:14:00,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:05,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:07,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:10,263 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:14:11,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:13,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:14:15,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:14:17,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:19,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:23,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:25,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:29,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:14:30,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:14:30,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:30,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:30,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:30,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:31,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:14:31,329 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 15:14:31,555 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 15:14:31,635 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 15:14:32,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 15:14:32,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:33,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:35,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:37,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:40,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:14:41,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:43,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:45,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:45,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:14:45,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:14:47,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:49,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:51,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:53,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:55,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:57,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:14:59,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:00,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:15:00,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:15:01,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:03,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:05,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:07,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:09,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:10,430 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:15:11,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:13,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:13,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:14,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:15,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:15:16,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:15:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:16,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:17,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:17,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:18,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:19,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:21,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:22,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:23,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:26,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:15:31,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:15:35,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:37,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:37,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:39,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:39,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:40,506 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:15:43,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:44,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:46,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:15:46,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:15:47,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:47,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:47,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:51,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:53,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:53,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:15:55,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:15:59,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:01,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:01,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:16:01,853 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:16:05,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:10,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:10,582 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:16:12,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:16,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:16:17,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:16:18,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:20,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:24,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:26,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:28,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:16:30,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:32,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:16:32,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:16:34,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:36,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:38,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:40,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:40,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:16:42,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:44,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:46,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:47,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:16:47,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:16:48,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:50,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:52,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:54,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:56,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:16:58,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:00,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:02,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:02,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:17:02,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:17:04,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:06,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:08,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:09,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:10,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:10,733 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:17:12,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:12,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:13,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:14,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:14,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:15,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:16,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:16,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:17:18,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:17:18,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:18,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:18,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:20,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:22,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:23,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:24,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:25,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:26,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:32,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:32,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:17:33,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:17:33,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:33,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:38,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:38,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:39,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:40,809 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:17:42,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:44,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:46,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:46,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:48,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:17:48,720 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:17:50,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:52,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:52,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:54,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:57,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:58,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:17:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:17:59,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:02,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:03,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:18:03,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:18:04,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:08,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:10,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:10,888 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:18:14,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:16,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:18,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:19,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:18:19,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:18:22,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:24,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:26,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:27,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:18:28,333 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 15:18:28,539 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 15:18:28,614 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 15:18:29,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 15:18:29,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:30,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:32,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:34,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:34,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:18:34,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:18:36,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:38,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:40,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:40,980 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:18:42,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:44,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:46,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:48,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:49,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:18:49,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:18:50,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:52,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:54,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:56,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:18:58,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:00,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:02,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:04,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:05,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:19:05,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:19:06,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:08,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:10,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:10,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:10,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:11,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:19:12,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:12,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:13,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:13,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:15,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:16,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:16,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:18,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:20,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:20,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:19:20,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:19:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:24,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:26,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:28,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:29,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:31,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:32,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:19:35,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:19:35,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:35,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:37,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:37,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:39,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:40,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,137 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:19:41,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:41,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:43,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:44,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:44,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:47,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:48,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:48,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:50,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:19:50,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:19:51,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:52,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:53,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:56,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:57,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:19:58,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:19:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:01,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:03,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:06,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:20:06,037 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:20:07,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:09,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:11,212 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:20:11,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:15,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:17,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:19,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:20,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:20,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:20:21,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:20:21,311 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:20:21,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:25,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:27,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:29,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:31,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:33,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:35,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:36,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:20:36,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:20:37,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:39,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:41,287 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:20:41,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:43,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:45,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:47,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:49,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:51,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:51,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:20:51,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:20:53,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:56,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:20:58,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:00,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:02,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:04,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:06,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:06,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:07,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:21:07,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:21:08,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:08,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:09,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:09,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:10,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:11,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:11,362 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:21:12,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:13,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:13,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:14,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:15,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:20,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:21,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:21:22,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:21:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:26,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:27,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:30,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:34,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:35,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:36,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:37,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:21:37,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:21:38,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,440 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:21:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:42,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:42,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:44,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:46,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:48,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:51,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:52,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:21:52,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:21:54,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:21:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:54,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:55,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:21:58,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:00,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:04,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:06,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:22:08,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:22:08,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:11,515 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:22:12,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:14,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:14,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:14,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:22:15,637 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 15:22:15,857 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 15:22:15,920 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 15:22:16,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 15:22:16,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:17,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:20,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:22,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:23,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:22:23,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:22:24,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:26,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:28,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:30,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:32,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:35,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:37,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:38,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:22:38,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:22:39,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:41,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:41,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:22:43,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:45,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:47,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:49,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:51,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:53,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:53,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:22:53,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:22:55,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:57,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:22:59,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:01,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:02,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:02,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:03,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:04,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:04,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:05,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:07,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:09,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:23:09,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:23:11,674 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:23:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:13,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:15,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:18,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:18,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:21,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:22,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:23:24,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:23:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:24,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:25,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:26,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:26,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:29,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:30,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:32,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:33,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:33,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:34,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:37,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:37,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:39,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:23:39,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:23:39,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:39,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:40,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:41,746 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:23:43,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:43,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:44,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:44,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:47,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:49,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:49,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:51,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:53,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:53,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:23:54,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:23:54,890 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:23:55,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:23:57,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:01,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:03,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:05,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:07,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:08,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:08,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:08,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:08,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:10,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:24:10,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:24:11,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:11,818 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:24:13,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:15,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:17,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:19,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:21,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:23,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:25,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:24:25,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:24:25,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:28,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:30,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:32,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:34,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:36,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:38,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:40,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:40,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:24:40,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:24:41,894 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:24:42,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:44,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:46,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:48,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:50,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:52,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:54,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:55,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:24:55,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:24:56,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:24:57,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:57,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:24:58,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:00,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:02,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:08,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:11,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:25:11,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:25:11,969 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:25:12,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:16,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:18,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:20,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:24,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:25,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:26,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:25:26,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:25:26,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:28,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:29,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:30,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:31,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:32,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:33,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:35,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:36,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:38,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:38,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:41,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:25:41,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:25:41,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,049 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:25:42,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:42,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:42,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:44,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:47,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:48,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:48,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:50,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:53,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:54,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:56,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:25:56,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:25:57,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:58,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:25:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:25:59,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:00,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:04,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:04,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 15:26:04,772 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 15:26:04,988 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 15:26:05,051 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 15:26:05,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 15:26:06,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:08,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:10,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:12,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:26:12,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:26:12,333 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:26:12,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:14,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:16,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:18,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:20,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:22,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:24,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:26,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:27,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:26:27,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:26:28,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:30,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:32,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:34,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:36,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:39,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:41,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:42,260 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:26:42,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:26:42,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:26:43,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:45,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:47,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:49,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:51,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:53,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:55,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:57,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:26:57,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:26:57,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:26:59,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:01,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:03,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:05,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:07,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:09,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:11,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:12,332 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:27:13,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:27:13,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:27:13,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:15,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:17,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:19,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:21,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:23,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:25,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:27,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:27:28,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:27:29,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:31,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:33,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:35,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:37,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:39,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:41,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:42,407 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:27:43,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:27:43,797 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:27:45,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:47,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:49,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:51,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:53,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:55,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:57,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:27:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:27:59,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:27:59,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:01,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:03,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:05,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:07,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:09,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:11,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:12,478 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:28:13,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:28:14,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:28:15,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:18,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:20,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:22,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:24,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:26,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:28,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:28:29,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:28:30,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:32,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:34,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:36,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:38,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:40,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:42,554 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:28:42,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:44,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:28:44,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:28:46,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:48,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:50,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:52,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:54,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:56,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:28:58,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:29:00,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:29:00,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:02,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:04,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:06,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:08,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:10,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:12,633 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:29:12,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:14,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:15,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:29:15,572 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:29:16,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:18,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:20,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:22,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:24,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:26,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:28,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:30,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:30,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:29:30,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:29:32,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:34,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:36,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:38,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:40,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:42,703 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:29:42,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:44,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:29:46,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:29:46,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:48,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:50,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:52,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:54,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:56,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:29:58,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:00,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:01,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:30:01,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:30:02,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:04,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:06,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:08,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:10,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:12,760 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:30:13,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:15,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:16,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:30:16,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:30:17,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:19,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:21,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:23,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:25,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:27,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:29,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:31,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:31,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:30:31,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:30:33,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:35,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:37,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:39,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:41,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:42,834 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:30:43,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:45,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:30:47,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:30:47,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:49,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:51,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:53,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:55,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:57,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:30:59,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:01,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:02,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:31:02,447 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:31:03,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:05,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:07,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:09,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:11,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:12,910 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:31:13,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:15,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:17,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:31:17,681 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:31:17,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:19,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:21,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:23,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:25,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:27,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:29,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:31,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:32,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:31:32,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:31:33,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:35,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:37,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:39,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:41,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:42,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:31:43,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:45,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:47,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:31:48,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:31:50,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:52,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:54,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:56,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:31:58,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:00,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:02,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:32:03,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:32:04,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:06,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:08,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:10,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:12,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:13,055 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:32:14,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:16,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:18,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:18,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:32:18,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:32:20,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:22,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:24,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:26,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:28,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:30,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:32,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:34,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:32:34,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:32:34,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:36,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:38,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:40,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:42,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:43,127 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:32:44,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:46,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:48,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:32:49,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:32:50,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:52,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:54,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:56,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:32:58,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:00,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:02,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:04,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:33:04,758 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:33:04,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:06,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:08,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:10,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:12,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:13,202 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:33:14,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:16,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:18,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:33:20,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:33:20,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:22,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:24,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:26,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:28,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:30,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:32,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:34,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:33:35,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:33:36,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:38,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:40,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:42,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:43,278 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:33:44,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:46,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:48,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:33:50,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:33:50,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:52,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:54,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:56,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:33:58,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:00,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:03,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:05,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:34:05,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:34:07,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:09,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:11,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:13,353 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:34:13,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:15,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:17,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:19,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:34:21,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:34:21,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:23,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:25,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:27,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:29,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:31,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:33,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:35,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:34:36,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:34:37,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:39,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:41,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:43,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:45,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:47,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:47,968 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:34:49,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:51,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:53,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:55,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:34:56,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:34:57,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:34:59,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:01,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:03,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:05,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:07,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:09,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:35:11,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:35:11,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:13,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:35:13,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:15,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:17,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:19,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:21,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:23,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:25,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:27,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:35:27,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:35:27,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:29,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:31,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:33,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:35,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:37,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:39,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:41,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:42,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:35:42,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:35:43,573 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:35:43,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:45,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:47,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:49,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:51,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:53,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:55,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:35:57,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:35:57,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:35:58,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:00,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:02,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:04,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:06,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:08,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:10,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:12,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:36:13,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:36:13,629 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:36:14,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:16,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:18,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:20,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:22,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:24,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:26,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:36:28,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:36:28,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:30,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:32,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:34,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:36,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:38,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:40,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:42,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:43,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:36:43,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:36:43,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:36:44,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:46,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:48,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:50,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:52,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:54,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:56,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:36:58,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:36:58,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:36:58,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:00,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:02,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:04,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:06,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:08,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:10,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:12,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:13,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:37:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:37:14,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:37:14,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:16,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:19,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:21,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:23,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:25,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:27,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:37:29,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:37:29,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:31,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:33,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:35,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:37,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:39,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:41,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:43,845 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:37:43,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:37:44,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:37:45,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:47,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:49,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:51,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:53,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:55,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:57,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:59,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:37:59,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:37:59,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:38:01,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:03,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:05,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:07,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:09,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:11,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:13,902 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:38:13,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:15,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:38:15,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:38:15,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:17,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:19,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:21,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:23,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:25,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:27,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:29,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:30,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:38:30,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:38:31,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:33,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:35,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:37,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:39,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:41,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:43,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:43,961 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:38:45,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:38:45,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:38:45,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:47,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:49,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:51,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:53,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:55,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:57,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:38:59,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:39:01,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:39:02,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:04,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:06,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:08,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:10,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:12,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:14,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:39:14,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:16,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:39:16,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:39:16,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:18,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:20,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:22,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:24,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:26,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:28,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:30,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:39:31,713 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:39:32,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:34,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:36,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:38,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:40,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:42,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:44,103 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:39:44,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:46,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:39:46,963 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:39:46,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:48,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:50,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:52,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:54,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:56,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:39:58,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:00,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:02,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:40:02,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:40:02,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:04,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:06,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:08,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:10,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:12,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:14,178 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:40:14,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:16,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:17,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:40:17,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:40:18,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:20,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:22,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:24,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:26,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:28,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:30,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:32,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:40:32,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:40:32,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:34,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:36,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:38,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:40,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:43,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:44,252 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:40:45,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:47,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:40:48,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:40:49,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:51,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:53,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:55,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:40:57,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:00,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:02,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:41:03,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:41:04,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:06,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:08,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:10,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:12,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:14,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:14,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:41:16,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:18,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:18,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:41:18,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:41:20,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:22,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:24,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:26,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:28,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:30,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:32,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:34,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:41:34,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:41:36,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:38,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:40,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:42,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:44,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:44,385 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:41:46,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:48,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:49,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:41:49,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:41:50,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:52,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:54,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:56,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:41:58,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:00,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:02,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:04,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:04,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:42:04,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:42:06,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:08,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:10,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:12,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:14,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:14,456 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:42:16,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:18,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:20,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:42:20,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:42:22,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:24,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:26,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:28,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:30,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:32,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:34,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:35,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:42:35,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:42:36,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:38,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:40,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:42,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:44,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:44,520 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:42:46,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:48,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:50,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:42:50,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:42:52,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:54,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:56,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:42:58,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:00,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:02,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:04,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:05,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:43:05,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:43:06,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:08,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:10,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:12,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:14,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:14,592 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:43:16,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:18,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:20,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:43:21,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:43:23,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:25,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:27,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:29,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:31,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:33,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:35,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:36,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:43:36,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:43:37,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:39,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:41,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:43,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:44,664 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:43:45,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:47,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:49,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:51,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:43:51,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:43:53,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:55,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:57,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:43:59,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:01,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:03,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:05,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:07,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:44:07,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:44:07,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:09,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:11,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:13,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:14,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:44:15,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:17,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:19,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:21,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:44:22,229 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:44:24,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:26,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:28,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:30,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:32,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:34,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:36,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:37,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:44:37,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:44:38,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:40,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:42,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:44,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:44,812 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:44:46,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:48,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:50,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:52,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:52,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:44:52,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:44:54,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:56,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:44:58,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:00,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:02,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:04,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:06,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:45:08,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:45:08,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:10,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:12,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:14,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:14,887 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:45:16,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:18,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:20,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:22,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:23,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:45:23,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:45:24,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:26,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:28,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:30,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:32,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:34,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:36,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:38,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:38,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:45:38,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:45:40,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:42,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:44,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:44,961 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:45:46,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:48,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:50,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:52,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:53,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:45:53,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:45:54,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:56,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:45:58,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:00,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:02,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:04,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:06,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:08,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:09,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:46:09,138 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:46:10,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:13,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:15,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:46:15,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:17,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:19,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:21,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:23,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:24,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:46:24,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:46:25,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:27,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:29,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:31,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:33,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:35,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:37,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:39,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:39,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:46:39,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:46:41,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:43,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:45,113 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:46:45,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:47,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:49,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:51,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:53,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:46:54,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:46:55,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:57,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:46:59,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:01,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:03,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:05,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:07,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:09,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:47:10,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:47:11,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:13,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:15,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:15,187 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:47:17,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:19,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:21,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:23,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:25,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:47:25,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:47:27,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:29,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:31,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:33,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:35,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:37,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:39,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:41,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:47:41,002 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:47:41,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:43,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:45,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:45,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:47:47,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:49,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:51,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:53,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:55,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:56,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:47:56,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:47:57,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:47:59,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:01,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:04,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:06,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:08,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:10,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:48:11,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:48:12,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:14,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:15,335 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:48:16,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:18,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:20,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:22,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:24,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:26,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:26,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:48:26,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:48:28,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:30,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:32,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:34,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:36,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:38,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:40,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:42,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:42,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:48:42,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:48:44,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:45,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:48:46,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:48,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:50,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:52,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:54,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:56,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:48:57,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:48:57,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:48:58,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:00,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:02,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:04,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:06,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:08,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:10,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:12,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:12,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:49:12,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:49:14,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:15,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:49:16,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:18,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:20,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:22,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:24,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:26,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:49:27,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:49:28,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:30,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:32,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:34,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:36,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:38,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:40,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:42,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:43,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:49:43,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:49:44,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:45,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:49:47,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:49,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:51,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:53,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:55,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:57,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:49:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:49:58,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:49:59,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:01,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:03,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:05,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:07,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:09,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:11,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:13,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:13,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:50:13,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:50:15,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:15,636 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:50:17,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:19,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:21,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:23,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:25,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:27,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:29,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:50:29,156 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:50:29,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:31,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:33,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:35,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:37,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:39,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:41,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:43,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:44,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:50:44,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:50:45,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:45,699 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:50:47,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:49,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:51,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:53,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:55,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:57,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:59,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:50:59,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:50:59,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:51:01,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:03,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:05,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:07,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:10,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:12,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:14,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:15,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:51:15,103 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:51:15,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:51:16,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:18,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:20,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:22,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:24,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:26,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:28,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:30,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:30,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:51:30,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:51:32,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:34,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:36,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:38,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:40,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:42,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:44,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:45,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:51:45,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:51:45,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:51:46,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:48,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:50,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:52,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:54,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:56,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:51:58,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:00,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:00,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:52:00,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:52:02,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:04,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:06,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:08,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:10,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:12,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:14,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:15,914 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:52:16,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:52:16,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:52:16,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:18,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:20,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:22,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:24,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:26,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:28,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:30,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:52:31,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:52:32,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:34,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:36,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:38,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:40,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:42,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:44,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:45,990 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:52:46,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:46,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:52:46,553 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:52:48,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:50,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:52,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:54,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:56,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:52:58,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:00,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:01,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:53:01,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:53:02,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:04,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:06,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:09,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:11,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:13,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:15,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:16,065 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:53:17,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:53:17,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:53:17,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:19,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:21,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:23,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:25,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:27,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:29,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:31,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:32,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:53:32,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:53:33,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:35,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:37,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:39,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:41,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:43,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:45,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:46,140 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:53:47,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:47,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:53:47,686 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:53:49,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:51,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:53,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:55,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:57,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:53:59,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:01,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:03,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:54:03,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:54:03,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:05,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:08,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:10,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:12,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:14,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:16,216 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:54:16,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:18,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:54:18,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:54:18,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:20,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:22,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:24,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:26,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:28,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:30,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:32,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:54:33,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:54:34,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:36,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:38,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:40,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:42,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:44,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:46,292 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:54:46,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:48,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:54:48,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:54:50,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:52,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:54,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:56,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:54:58,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:00,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:02,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:04,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:55:04,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:55:04,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:06,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:08,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:10,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:12,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:14,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:16,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:55:16,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:18,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:55:19,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:55:20,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:22,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:24,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:26,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:28,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:30,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:32,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:34,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:34,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:55:34,658 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:55:37,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:39,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:41,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:43,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:45,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:46,440 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:55:47,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:49,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:55:49,881 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:55:51,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:53,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:55,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:57,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:55:59,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:01,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:03,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:05,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:56:05,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:56:05,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:07,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:09,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:11,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:13,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:15,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:16,515 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:56:17,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:19,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:20,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:56:20,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:56:21,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:23,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:25,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:27,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:29,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:31,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:33,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:35,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:35,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:56:35,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:56:37,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:39,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:41,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:43,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:45,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:46,589 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:56:47,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:49,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:50,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:56:50,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:56:51,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:53,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:56,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:56:58,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:00,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:02,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:04,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:57:06,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:57:06,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:08,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:10,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:12,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:14,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:16,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:16,664 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:57:18,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:20,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:21,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:57:21,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:57:22,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:24,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:26,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:28,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:30,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:32,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:34,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:36,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:36,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:57:36,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:57:38,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:40,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:42,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:44,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:46,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:46,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:57:48,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:50,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:52,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:57:52,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:57:52,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:54,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:56,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:57:58,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:00,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:02,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:04,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:06,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:07,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:58:07,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:58:08,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:11,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:13,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:15,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:16,820 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:58:17,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:19,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:21,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:58:22,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:58:23,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:25,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:27,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:29,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:31,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:33,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:35,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:37,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:37,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:58:37,853 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:58:39,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:41,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:43,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:45,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:46,895 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:58:47,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:49,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:51,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:53,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:58:53,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:58:53,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:55,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:57,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:58:59,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:01,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:03,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:05,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:07,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:08,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:59:08,392 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:59:09,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:11,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:13,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:15,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:16,971 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:59:17,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:19,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:21,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:23,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:59:23,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:59:25,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:27,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:29,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:32,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:34,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:36,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:38,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:38,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:59:38,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:59:40,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:42,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:44,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:46,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:47,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 15:59:48,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:50,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:52,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:54,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 15:59:54,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 15:59:54,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:56,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 15:59:58,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:00,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:02,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:04,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:06,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:08,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:09,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:00:09,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:00:10,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:12,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:14,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:16,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:17,122 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:00:18,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:20,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:22,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:24,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:24,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:00:24,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:00:26,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:28,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:30,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:32,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:34,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:36,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:38,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:00:40,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:00:40,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:42,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:44,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:46,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:47,189 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:00:48,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:50,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:52,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:54,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:55,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:00:55,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:00:57,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:00:59,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:01,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:03,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:05,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:07,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:09,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:10,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:01:10,867 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:01:11,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:13,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:15,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:17,253 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:01:17,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:19,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:21,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:23,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:25,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:26,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:01:26,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:01:27,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:29,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:31,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:33,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:35,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:37,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:39,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:41,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:41,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:01:41,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:01:43,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:45,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:47,320 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:01:47,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:49,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:51,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:53,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:55,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:01:56,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:01:57,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:01:59,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:01,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:03,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:05,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:07,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:09,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:11,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:12,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:02:12,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:02:13,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:15,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:17,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:02:17,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:20,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:22,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:24,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:26,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:27,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:02:27,565 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:02:28,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:30,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:32,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:34,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:36,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:38,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:40,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:42,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:42,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:02:42,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:02:44,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:46,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:47,454 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:02:48,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:50,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:52,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:54,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:56,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:02:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:02:58,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:02:58,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:00,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:02,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:04,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:06,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:08,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:10,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:12,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:03:13,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:03:14,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:16,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:17,524 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:03:18,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:20,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:22,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:24,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:26,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:28,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:28,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:03:28,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:03:30,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:32,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:34,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:37,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:39,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:41,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:43,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:43,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:03:43,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:03:45,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:47,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:47,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:03:49,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:51,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:53,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:55,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:57,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:03:59,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:03:59,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:03:59,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:01,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:03,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:05,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:07,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:09,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:11,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:13,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:14,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:04:14,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:04:15,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:17,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:17,676 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:04:19,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:21,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:23,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:25,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:27,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:29,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:30,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:04:30,344 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:04:31,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:33,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:35,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:37,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:39,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:41,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:43,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:45,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:45,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:04:45,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:04:47,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:47,754 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:04:49,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:51,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:53,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:55,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:57,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:04:59,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:00,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:05:00,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:05:01,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:04,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:06,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:08,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:10,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:12,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:14,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:16,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:05:16,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:05:16,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:17,823 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:05:18,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:20,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:22,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:24,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:26,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:28,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:30,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:05:31,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:05:32,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:34,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:36,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:38,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:40,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:42,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:44,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:46,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:46,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:05:46,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:05:47,897 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:05:48,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:50,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:52,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:54,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:56,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:05:58,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:00,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:02,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:06:02,233 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:06:02,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:04,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:06,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:08,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:10,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:12,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:14,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:16,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:17,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:06:17,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:06:17,971 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:06:18,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:20,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:22,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:24,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:26,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:28,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:30,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:32,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:32,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:06:32,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:06:35,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:37,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:39,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:41,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:43,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:45,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:47,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:47,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:06:47,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:06:48,214 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:06:49,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:51,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:53,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:55,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:57,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:06:59,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:01,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:07:03,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:07:03,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:05,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:07,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:09,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:11,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:13,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:15,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:17,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:18,119 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:07:18,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:07:18,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:07:19,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:21,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:23,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:25,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:27,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:29,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:31,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:33,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:33,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:07:33,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:07:35,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:37,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:39,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:41,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:43,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:46,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:48,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:07:48,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:49,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:07:49,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:07:50,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:52,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:54,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:56,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:07:58,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:00,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:02,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:08:04,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:08:04,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:06,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:08,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:10,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:12,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:14,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:16,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:18,255 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:08:18,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:08:19,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:08:20,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:08:34,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:08:34,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:08:48,331 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:08:50,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:08:50,102 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:09:05,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:09:05,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:09:18,407 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:09:20,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:09:20,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:09:21,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:09:21,191 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 16:09:21,233 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 16:09:21,310 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 16:09:21,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 16:09:21,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:09:22,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:09:28,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:09:36,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:09:36,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:09:44,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:09:48,493 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:09:51,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:09:51,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:10:06,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:10:06,568 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:10:18,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:10:21,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:10:21,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:10:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:10:37,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:10:48,643 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:10:52,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:10:52,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:11:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:11:07,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:11:18,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:11:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:11:22,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:11:37,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:11:37,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:11:48,800 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:11:53,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:11:53,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:12:08,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:12:08,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:12:18,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:12:23,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:12:23,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:12:38,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:12:38,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:12:48,958 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:12:54,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:12:54,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:13:09,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:13:09,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:13:19,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:13:24,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:13:24,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:13:38,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:40,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:13:40,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:13:40,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:42,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:46,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:48,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:49,111 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:13:50,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:52,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:54,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:55,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:13:55,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:13:56,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:13:59,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:01,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:03,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:05,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:07,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:09,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:10,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:14:10,664 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:14:11,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:13,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:15,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:17,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:19,185 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:14:19,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:21,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:23,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:25,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:14:25,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:14:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:26,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:27,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:29,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:34,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:37,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:14:41,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:14:43,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:47,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:49,263 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:14:51,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:53,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:55,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:14:56,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:14:56,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:14:56,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:56,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:14:59,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:00,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:03,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:03,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:05,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:07,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:07,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:09,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:10,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:10,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:11,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:15:11,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:15:12,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:12,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:13,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:15,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:15,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:16,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:16,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:17,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:18,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:19,343 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:15:19,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:20,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:21,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:23,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:24,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:24,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:27,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:15:27,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:15:27,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:29,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:29,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:31,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:32,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:32,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:32,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:32,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:33,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:34,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:36,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:15:37,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:39,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:41,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:42,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:15:42,254 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:15:43,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:45,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:47,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:49,421 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:15:49,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:51,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:53,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:55,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:15:57,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:15:57,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:15:59,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:01,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:03,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:06,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:08,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:10,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:16:12,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:16:12,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:14,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:16,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:18,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:19,493 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:16:20,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:22,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:23,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:24,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:26,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:27,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:16:27,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:16:34,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:35,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:35,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:40,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:43,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:16:43,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:16:44,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:48,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:49,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:16:52,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:53,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:53,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:56,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:16:57,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:57,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:58,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:16:58,461 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:16:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:16:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:00,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:02,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:04,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:04,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:06,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:07,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:07,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:08,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:09,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:10,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:12,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:12,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:13,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:17:13,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:17:13,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:13,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:14,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:15,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:16,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:17,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:18,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:19,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:17:19,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:19,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:20,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:21,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:21,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:22,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:24,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:25,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:26,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:17:28,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:17:28,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:29,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:29,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:31,001 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 16:17:31,121 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 16:17:31,175 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 16:17:32,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 16:17:33,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:34,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:34,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:35,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:35,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:36,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:37,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:39,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:41,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:43,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:43,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:17:44,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:17:44,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:17:47,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:49,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:49,738 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:17:51,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:53,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:55,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:57,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:59,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:17:59,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:17:59,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:18:02,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:04,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:06,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:08,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:10,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:12,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:14,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:14,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:18:14,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:18:16,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:18,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:19,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:18:20,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:22,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:24,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:30,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:18:30,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:18:34,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:40,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:43,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:43,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:18:44,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:18:45,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:18:48,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:49,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:18:52,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:18:56,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:00,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:00,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:19:00,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:19:00,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:02,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:06,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:06,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:11,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:12,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:14,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:19:15,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:19:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:15,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:16,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:18,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:18,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:19,969 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:19:20,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:20,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:20,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:21,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:22,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:24,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:24,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:25,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:26,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:26,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:29,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:19:30,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:19:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:33,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:34,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:36,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:36,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:38,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:38,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:40,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:41,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:42,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:43,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:19:44,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:46,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:19:46,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:19:48,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:50,040 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:19:50,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:52,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:54,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:56,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:19:58,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:00,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:01,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:20:01,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:20:03,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:05,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:07,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:09,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:11,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:13,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:15,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:20:16,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:20:17,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:19,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:20,108 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:20:21,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:23,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:31,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:20:32,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:20:37,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:41,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:43,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:43,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:45,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:47,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:47,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:20:47,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:20:50,185 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:20:51,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:52,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:52,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:20:55,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:20:59,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:01,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:21:02,607 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:21:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:03,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:04,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:05,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:05,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:06,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:07,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:08,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:11,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:12,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:13,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:16,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:17,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:21:17,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:21:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:18,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,260 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:21:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:20,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:21,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:23,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:24,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:25,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:25,190 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 16:21:25,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:25,415 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 16:21:25,487 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 16:21:26,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:26,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 16:21:26,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:27,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:27,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:28,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:28,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:29,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:29,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:31,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:31,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:32,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:21:33,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:21:33,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:34,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:36,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:39,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:40,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:40,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:41,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:43,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:43,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:43,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:21:45,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:47,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:21:48,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:21:49,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:50,349 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:21:51,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:53,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:55,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:57,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:21:59,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:01,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:03,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:03,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:22:03,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:22:05,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:07,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:09,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:11,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:13,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:15,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:17,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:18,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:22:18,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:22:20,425 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:22:27,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:33,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:34,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:22:34,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:22:37,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:41,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:45,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:49,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:22:49,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:22:49,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:49,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:50,499 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:22:52,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:56,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:22:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:22:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:00,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:02,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:23:04,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:23:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:05,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:06,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:07,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:08,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:11,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:12,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:12,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:14,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:15,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:15,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:16,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:16,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:17,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:18,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:19,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:23:19,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:23:20,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:20,574 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:23:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:21,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:22,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:23,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:24,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:24,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:26,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:26,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:28,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:29,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:30,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:32,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:34,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:35,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:23:35,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:23:36,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:36,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:38,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:38,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:39,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:40,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:42,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:44,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:46,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:23:48,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:50,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:23:50,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:23:50,700 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:23:52,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:54,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:56,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:23:58,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:00,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:02,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:04,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:05,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:24:05,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:24:06,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:08,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:10,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:12,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:14,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:20,723 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:24:21,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:24:21,273 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:24:22,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:28,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:30,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:34,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:36,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:24:36,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:24:38,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:42,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:47,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:49,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:50,798 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:24:51,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:24:51,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:24:53,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:53,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:24:55,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:24:59,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:01,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:02,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:02,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:05,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:06,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:25:07,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:25:07,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:07,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:07,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:10,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:11,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:11,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:12,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:13,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:14,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:14,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:15,860 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 16:25:15,967 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 16:25:16,023 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 16:25:16,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 16:25:16,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:16,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:17,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:18,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:20,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:25:21,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:21,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:21,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:25:22,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:25:22,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:22,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:23,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:23,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:24,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:25,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:26,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:26,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:27,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:28,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:29,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:30,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:30,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:31,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:32,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:33,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:35,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:35,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:36,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:37,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:25:37,566 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:25:39,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:39,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:41,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:43,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:45,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:46,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:25:47,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:49,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:50,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:25:51,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:52,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:25:52,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:25:53,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:55,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:57,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:25:59,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:01,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:03,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:05,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:07,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:08,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:26:08,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:26:17,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:21,026 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:26:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:26:23,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:26:23,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:27,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:31,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:35,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:38,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:26:38,497 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:26:39,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:41,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:45,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:49,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:51,103 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:26:51,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:53,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:26:53,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:26:53,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:26:55,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:55,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:26:58,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:00,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:04,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:05,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:06,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:07,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:07,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:08,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:27:08,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:27:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:10,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:11,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:13,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:14,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:14,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:15,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:16,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:19,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:20,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:20,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,178 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:27:21,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:21,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:27:24,367 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:27:24,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:25,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:27,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:28,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:29,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:29,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:30,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:31,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:31,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:32,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:33,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:33,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:34,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:36,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:38,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:39,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:27:39,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:27:40,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:42,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:44,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:46,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:48,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:48,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:27:50,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:51,253 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:27:52,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:54,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:54,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:27:54,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:27:56,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:27:58,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:00,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:02,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:04,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:10,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:28:10,110 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:28:12,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:18,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:21,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:28:22,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:28:25,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:28:26,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:30,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:34,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:38,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:40,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:40,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:28:40,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:28:44,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:46,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:50,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:51,404 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:28:52,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:55,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:55,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:56,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:28:56,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:28:56,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:28:58,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:28:58,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:00,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:04,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:04,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:06,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:07,477 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 16:29:07,687 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 16:29:07,760 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 16:29:08,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:08,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 16:29:08,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:09,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:09,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:10,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:10,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:29:11,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:29:13,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:13,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:14,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:16,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:17,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:17,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:18,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:20,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:20,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:21,797 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:29:22,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:25,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:26,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:26,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:29:26,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:29:27,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:27,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:28,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:30,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:32,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:33,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:35,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:36,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:38,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:39,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:39,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:40,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:41,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:29:41,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:29:42,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:44,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:46,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:49,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:29:49,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:51,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:51,874 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:29:53,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:55,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:29:57,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:29:57,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:29:59,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:09,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:30:12,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:30:15,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:19,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:21,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:21,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:30:25,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:27,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:30:27,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:30:29,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:33,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:37,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:39,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:30:43,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:30:43,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:45,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:49,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:51,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:52,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:30:55,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:57,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:30:57,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:57,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:30:58,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:30:58,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:30:59,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:03,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:05,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:06,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:07,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:07,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:09,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:11,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:11,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:13,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:31:14,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:31:14,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:14,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:15,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:16,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:17,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:17,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:19,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:19,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:21,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:21,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,096 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:31:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:22,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:23,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:25,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:26,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:27,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:28,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:28,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:31:29,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:31:29,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:29,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:31,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:32,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:33,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:33,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:35,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:37,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:38,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:39,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:41,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:43,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:44,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:31:44,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:31:45,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:47,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:48,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:48,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:31:49,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:51,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:52,171 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:31:54,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:31:59,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:31:59,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:32:02,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:08,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:12,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:32:15,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:32:16,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:20,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:22,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:32:24,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:28,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:30,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:32:30,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:32:30,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:34,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:36,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:40,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:42,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:45,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:32:45,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:32:46,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:48,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:50,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:52,327 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:32:54,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:56,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:32:56,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:56,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:57,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 16:32:57,192 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 16:32:57,403 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 16:32:57,460 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 16:32:57,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 16:32:58,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:00,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:01,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:33:01,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:33:02,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:04,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:06,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:08,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:10,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:13,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:15,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:16,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:33:16,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:33:17,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:19,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:21,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:22,424 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:33:23,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:25,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:27,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:29,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:31,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:31,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:33:31,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:33:33,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:35,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:37,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:39,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:41,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:43,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:45,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:47,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:33:47,012 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:33:47,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:49,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:51,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:52,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:33:53,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:55,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:57,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:33:59,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:01,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:02,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:34:02,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:34:03,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:05,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:07,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:09,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:11,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:13,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:15,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:34:17,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:34:17,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:19,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:21,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:22,571 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:34:24,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:26,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:28,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:30,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:32,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:32,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:34:32,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:34:34,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:36,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:38,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:40,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:42,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:44,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:46,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:48,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:34:48,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:34:48,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:50,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:52,650 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:34:52,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:54,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:56,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:34:58,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:00,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:02,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:03,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:35:03,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:35:04,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:06,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:08,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:10,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:12,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:14,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:16,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:18,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:18,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:35:18,852 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:35:20,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:22,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:22,727 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:35:24,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:26,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:28,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:30,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:32,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:34,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:35:34,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:35:34,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:36,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:38,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:40,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:42,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:44,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:46,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:48,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:49,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:35:49,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:35:50,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:52,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:35:53,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:55,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:57,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:35:59,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:01,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:03,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:04,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:36:04,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:36:05,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:07,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:09,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:11,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:13,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:15,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:17,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:19,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:19,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:36:19,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:36:21,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:22,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:36:23,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:25,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:27,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:29,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:31,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:33,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:35,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:36:35,208 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:36:35,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:37,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:39,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:41,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:43,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:45,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:47,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:49,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:50,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:36:50,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:36:51,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:52,952 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:36:53,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:55,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:57,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:36:59,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:01,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:03,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:05,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:37:05,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:37:05,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:07,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:09,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:11,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:13,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:15,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:18,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:20,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:37:21,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:37:22,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:23,026 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:37:24,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:26,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:28,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:30,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:32,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:34,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:36,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:37:36,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:37:36,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:38,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:40,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:42,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:44,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:46,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:48,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:50,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:37:51,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:37:52,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:53,102 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:37:54,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:56,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:37:58,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:00,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:02,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:04,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:06,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:07,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:38:07,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:38:08,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:10,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:12,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:14,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:16,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:18,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:20,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:38:22,384 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:38:22,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:23,176 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:38:24,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:26,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:28,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:30,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:32,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:34,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:36,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:37,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:38:37,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:38:38,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:40,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:42,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:44,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:46,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:48,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:50,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:52,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:52,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:38:52,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:38:53,249 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:38:54,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:56,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:38:58,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:00,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:02,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:04,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:06,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:39:08,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:39:08,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:10,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:12,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:14,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:16,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:19,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:21,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:23,318 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:39:23,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:39:23,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:39:23,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:25,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:27,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:29,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:31,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:33,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:35,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:37,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:38,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:39:38,819 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:39:39,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:41,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:43,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:45,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:47,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:49,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:51,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:53,390 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:39:53,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:54,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:39:54,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:39:55,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:57,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:39:59,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:01,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:03,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:05,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:07,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:09,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:40:09,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:40:09,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:11,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:13,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:15,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:17,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:19,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:21,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:23,466 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:40:23,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:40:24,653 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:40:25,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:27,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:29,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:31,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:33,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:35,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:37,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:39,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:40:39,884 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:40:41,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:43,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:45,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:47,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:49,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:51,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:53,540 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:40:53,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:55,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:40:55,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:40:55,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:57,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:40:59,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:01,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:03,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:05,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:07,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:10,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:41:10,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:41:10,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:12,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:14,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:16,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:18,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:20,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:22,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:23,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:41:24,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:25,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:41:25,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:41:26,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:28,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:30,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:32,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:34,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:36,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:38,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:40,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:41,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:41:41,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:41:42,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:44,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:46,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:48,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:50,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:52,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:53,693 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:41:54,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:56,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:41:56,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:41:56,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:41:58,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:00,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:02,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:04,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:06,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:08,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:10,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:11,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:42:11,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:42:12,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:14,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:16,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:18,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:20,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:22,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:23,770 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:42:24,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:26,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:27,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:42:27,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:42:28,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:30,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:32,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:34,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:36,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:38,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:40,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:42,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:42:42,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:42:42,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:44,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:46,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:48,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:50,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:53,844 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:42:53,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:55,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:42:57,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:42:57,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:42:59,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:01,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:03,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:05,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:07,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:09,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:11,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:43:12,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:43:13,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:15,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:17,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:19,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:21,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:23,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:23,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:43:25,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:27,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:28,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:43:28,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:43:29,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:31,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:33,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:35,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:37,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:39,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:41,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:43,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:43:43,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:43:43,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:45,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:47,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:49,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:51,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:53,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:53,991 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:43:55,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:57,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:43:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:43:58,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:43:59,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:01,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:03,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:05,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:07,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:09,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:12,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:14,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:44:14,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:44:14,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:16,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:18,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:20,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:22,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:24,065 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:44:24,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:26,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:28,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:44:29,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:44:30,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:32,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:34,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:36,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:38,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:40,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:42,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:44,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:44:44,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:44:44,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:46,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:48,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:50,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:52,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:54,140 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:44:54,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:56,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:44:58,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:00,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:45:00,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:45:01,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:03,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:05,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:07,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:09,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:11,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:13,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:15,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:45:15,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:45:15,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:17,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:19,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:21,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:23,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:24,214 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:45:25,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:27,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:29,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:45:30,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:45:31,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:33,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:35,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:37,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:39,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:41,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:43,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:45,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:45:45,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:45:45,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:47,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:49,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:51,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:53,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:54,287 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:45:55,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:57,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:45:59,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:01,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:46:01,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:46:01,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:03,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:05,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:07,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:09,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:11,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:13,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:15,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:16,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:46:16,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:46:17,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:19,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:21,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:23,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:24,363 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:46:25,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:27,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:29,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:46:31,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:46:31,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:33,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:35,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:37,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:39,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:42,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:44,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:46,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:46:46,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:46:46,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:48,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:50,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:52,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:54,436 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:46:54,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:56,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:46:58,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:00,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:02,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:47:02,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:47:02,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:04,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:06,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:08,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:10,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:13,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:15,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:17,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:17,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:47:17,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:47:19,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:21,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:23,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:24,517 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:47:25,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:27,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:29,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:31,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:47:32,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:47:33,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:35,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:37,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:39,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:41,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:43,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:45,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:47,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:47,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:47:47,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:47:49,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:51,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:53,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:54,592 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:47:55,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:57,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:47:59,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:01,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:03,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:03,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:48:03,158 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:48:05,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:07,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:09,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:11,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:13,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:15,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:17,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:18,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:48:18,388 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:48:19,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:21,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:23,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:24,667 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:48:25,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:27,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:29,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:31,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:33,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:33,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:48:33,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:48:35,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:37,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:39,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:41,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:43,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:45,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:48,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:48:48,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:48:50,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:52,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:54,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:54,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:48:56,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:48:58,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:00,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:02,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:04,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:04,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:49:04,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:49:06,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:08,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:10,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:12,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:14,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:16,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:18,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:49:19,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:49:20,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:22,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:24,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:24,819 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:49:26,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:28,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:30,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:32,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:34,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:49:34,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:49:36,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:38,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:40,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:42,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:44,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:46,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:48,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:50,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:49:50,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:49:50,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:52,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:54,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:54,898 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:49:56,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:49:58,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:00,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:03,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:05,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:05,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:50:05,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:50:07,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:09,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:11,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:13,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:15,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:17,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:19,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:50:20,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:50:21,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:23,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:24,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:50:25,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:27,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:29,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:31,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:33,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:35,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:35,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:50:35,918 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:50:37,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:39,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:41,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:43,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:45,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:47,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:49,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:51,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:51,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:50:51,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:50:53,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:55,050 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:50:55,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:57,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:50:59,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:01,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:03,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:05,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:06,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:51:06,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:51:07,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:09,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:11,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:13,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:15,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:17,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:19,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:21,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:21,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:51:21,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:51:23,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:25,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:25,127 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:51:27,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:29,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:31,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:33,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:35,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:37,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:51:37,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:51:38,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:40,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:42,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:44,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:46,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:48,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:50,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:52,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:51:52,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:51:54,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:55,203 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:51:56,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:51:58,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:00,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:02,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:04,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:06,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:07,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:52:07,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:52:08,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:10,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:12,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:14,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:16,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:18,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:20,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:22,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:22,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:52:22,867 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:52:24,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:25,280 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:52:26,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:28,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:30,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:32,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:34,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:36,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:38,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:52:38,094 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:52:38,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:40,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:42,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:44,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:46,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:48,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:50,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:52,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:53,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:52:53,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:52:54,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:55,353 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:52:56,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:52:58,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:00,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:02,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:04,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:06,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:08,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:08,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:53:08,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:53:10,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:12,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:14,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:16,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:18,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:20,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:22,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:24,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:53:24,028 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:53:24,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:25,418 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:53:26,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:28,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:30,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:32,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:34,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:36,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:38,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:39,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:53:39,254 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:53:41,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:43,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:45,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:47,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:49,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:51,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:53,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:53:54,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:53:55,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:55,486 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:53:57,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:53:59,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:01,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:03,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:05,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:07,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:09,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:09,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:54:09,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:54:11,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:13,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:15,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:17,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:19,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:21,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:23,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:25,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:54:25,212 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:54:25,564 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:54:27,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:29,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:31,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:33,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:35,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:37,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:39,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:54:40,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:54:41,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:43,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:45,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:47,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:49,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:51,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:53,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:55,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:55,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:54:55,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:54:55,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:54:57,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:54:59,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:01,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:03,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:05,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:07,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:09,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:55:11,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:55:11,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:14,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:16,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:18,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:20,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:22,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:24,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:25,699 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:55:26,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:26,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:55:26,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:55:28,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:30,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:32,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:34,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:36,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:38,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:40,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:41,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:55:41,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:55:42,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:44,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:46,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:48,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:50,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:52,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:54,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:55,774 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:55:56,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:55:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:55:57,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:55:58,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:00,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:02,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:04,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:06,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:08,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:10,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:12,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:12,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:56:12,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:56:14,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:16,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:18,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:20,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:22,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:24,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:25,848 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:56:26,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:56:27,523 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:56:28,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:30,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:32,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:34,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:36,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:38,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:40,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:42,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:42,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:56:42,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:56:44,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:46,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:48,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:50,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:52,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:54,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:55,924 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:56:56,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:56:58,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:56:58,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:56:59,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:01,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:03,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:05,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:07,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:09,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:11,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:13,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:57:14,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:57:15,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:17,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:19,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:21,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:23,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:25,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:26,000 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:57:27,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:29,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:29,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:57:29,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:57:31,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:33,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:35,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:37,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:39,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:41,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:43,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:57:44,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:57:45,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:47,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:49,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:51,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:53,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:55,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:57:56,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:57:57,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:58:00,106 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:58:00,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:02,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:04,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:06,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:08,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:10,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:12,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:14,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:15,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:58:15,329 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:58:16,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:18,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:20,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:22,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:24,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:26,152 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:58:26,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:28,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:30,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:30,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:58:30,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:58:32,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:34,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:36,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:38,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:40,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:42,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:44,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:45,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:58:45,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:58:46,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:48,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:50,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:52,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:54,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:56,228 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:58:56,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:58:58,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:00,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:01,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:59:01,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:59:02,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:04,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:06,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:08,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:10,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:12,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:14,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:16,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:16,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:59:16,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:59:18,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:20,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:22,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:24,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:26,304 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:59:26,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:28,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:30,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:59:31,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:59:32,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:34,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:36,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:38,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:40,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:42,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:44,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:46,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:47,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 16:59:47,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 16:59:49,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:51,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:53,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:55,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:56,383 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 16:59:57,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 16:59:59,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:01,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:02,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:00:02,590 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:00:03,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:05,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:07,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:09,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:11,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:13,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:15,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:17,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:17,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:00:17,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:00:19,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:21,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:23,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:25,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:26,458 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:00:27,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:29,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:31,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:00:33,079 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:00:33,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:35,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:37,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:39,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:41,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:43,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:45,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:47,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:48,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:00:48,371 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:00:49,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:51,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:53,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:56,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:00:56,534 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:00:58,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:00,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:02,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:01:03,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:01:04,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:06,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:08,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:10,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:12,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:14,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:16,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:18,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:18,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:01:18,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:01:20,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:22,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:24,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:26,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:26,604 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:01:28,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:30,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:32,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:34,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:01:34,158 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:01:34,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:36,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:38,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:40,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:42,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:44,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:46,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:48,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:49,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:01:49,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:01:50,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:52,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:54,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:56,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:01:56,678 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:01:58,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:00,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:02,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:04,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:04,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:02:04,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:02:06,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:08,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:10,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:12,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:14,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:16,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:19,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:02:19,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:02:21,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:23,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:25,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:26,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:02:27,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:29,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:31,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:33,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:35,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:02:35,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:02:35,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:37,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:39,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:41,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:43,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:45,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:47,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:49,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:02:50,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:02:51,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:53,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:55,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:56,829 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:02:57,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:02:59,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:01,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:03,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:05,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:05,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:03:05,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:03:07,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:09,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:11,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:13,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:15,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:17,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:19,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:21,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:03:21,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:03:21,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:23,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:25,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:26,907 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:03:27,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:29,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:31,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:33,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:35,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:36,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:03:36,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:03:37,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:39,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:41,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:43,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:45,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:47,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:49,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:51,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:51,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:03:51,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:03:53,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:55,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:03:56,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:03:58,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:00,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:02,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:04,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:06,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:07,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:04:07,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:04:08,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:10,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:12,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:14,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:16,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:18,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:20,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:22,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:04:22,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:04:22,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:24,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:26,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:27,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:04:28,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:30,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:32,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:34,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:36,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:37,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:04:37,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:04:38,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:40,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:42,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:44,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:46,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:48,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:50,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:52,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:52,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:04:52,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:04:55,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:57,135 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:04:57,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:04:59,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:01,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:03,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:05,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:07,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:08,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:05:08,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:05:09,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:11,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:13,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:15,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:17,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:19,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:21,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:23,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:05:23,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:05:23,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:25,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:27,211 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:05:27,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:29,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:31,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:33,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:35,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:37,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:38,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:05:38,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:05:39,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:41,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:43,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:45,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:47,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:49,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:51,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:53,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:54,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:05:54,052 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:05:55,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:57,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:05:57,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:05:59,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:01,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:04,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:06,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:08,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:09,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:06:09,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:06:10,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:12,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:14,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:16,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:18,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:20,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:22,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:24,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:24,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:06:24,602 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:06:26,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:27,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:06:28,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:30,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:32,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:34,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:36,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:38,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:39,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:06:39,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:06:40,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:42,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:44,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:46,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:48,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:50,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:52,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:54,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:06:55,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:06:56,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:06:57,442 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:06:58,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:00,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:02,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:04,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:06,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:08,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:10,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:07:10,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:07:10,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:12,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:14,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:16,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:18,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:20,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:22,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:24,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:25,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:07:25,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:07:26,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:27,519 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:07:28,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:30,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:32,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:34,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:36,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:38,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:40,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:40,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:07:40,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:07:42,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:44,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:46,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:48,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:50,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:53,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:55,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:56,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:07:56,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:07:57,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:07:57,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:07:59,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:01,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:03,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:05,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:07,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:09,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:11,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:08:11,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:08:11,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:13,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:15,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:17,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:19,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:21,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:23,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:25,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:08:26,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:08:27,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:27,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:08:29,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:31,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:33,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:35,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:37,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:39,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:41,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:08:42,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:08:43,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:45,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:47,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:49,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:51,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:53,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:55,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:57,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:08:57,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:08:57,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:08:57,730 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:08:59,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:01,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:03,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:05,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:08,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:10,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:12,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:12,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:09:12,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:09:14,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:16,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:18,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:20,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:22,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:24,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:26,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:27,796 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:09:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:09:27,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:09:28,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:30,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:32,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:34,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:36,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:38,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:40,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:42,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:09:43,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:09:44,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:46,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:48,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:50,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:52,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:54,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:56,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:09:57,872 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:09:58,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:09:58,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:09:58,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:00,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:02,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:04,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:06,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:08,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:10,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:12,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:13,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:10:13,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:10:14,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:16,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:18,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:20,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:22,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:24,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:26,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:27,946 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:10:28,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:29,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:10:29,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:10:30,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:32,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:34,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:36,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:39,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:41,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:43,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:10:44,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:10:45,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:47,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:49,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:51,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:53,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:55,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:57,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:58,017 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:10:59,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:10:59,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:10:59,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:11:01,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:03,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:05,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:07,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:09,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:11,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:13,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:14,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:11:14,899 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:11:15,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:17,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:19,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:21,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:23,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:25,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:27,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:28,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:11:29,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:11:30,183 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:11:31,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:33,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:35,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:37,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:39,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:41,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:43,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:45,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:11:45,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:11:45,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:47,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:49,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:51,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:53,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:55,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:57,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:11:58,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:11:59,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:00,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:12:00,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:12:02,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:04,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:06,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:08,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:10,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:12,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:14,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:16,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:12:16,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:12:16,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:18,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:20,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:22,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:24,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:26,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:28,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:12:28,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:30,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:31,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:12:31,414 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:12:32,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:34,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:36,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:38,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:40,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:42,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:44,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:46,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:12:46,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:12:46,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:48,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:50,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:52,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:54,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:56,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:12:58,305 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:12:58,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:00,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:01,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:13:01,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:13:02,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:04,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:06,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:08,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:10,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:12,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:14,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:16,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:17,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:13:17,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:13:18,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:20,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:22,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:24,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:26,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:28,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:13:28,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:30,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:13:32,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:13:32,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:34,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:36,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:38,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:41,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:43,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:45,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:47,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:13:47,672 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:13:47,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:49,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:51,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:53,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:55,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:57,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:13:58,435 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:13:59,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:01,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:02,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:14:02,889 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:14:03,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:05,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:07,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:09,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:11,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:13,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:15,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:17,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:18,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:14:18,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:14:19,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:21,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:23,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:25,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:27,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:28,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:14:29,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:31,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:14:33,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:14:33,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:35,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:37,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:39,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:41,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:43,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:45,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:47,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:48,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:14:48,720 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:14:49,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:51,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:53,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:55,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:57,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:14:58,567 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:14:59,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:15:01,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:15:04,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:15:04,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:15:19,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:15:19,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:15:28,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:15:34,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:15:34,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:15:49,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:15:49,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:15:58,717 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:16:02,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:16:02,994 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 17:16:03,047 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 17:16:03,125 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 17:16:03,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 17:16:03,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:16:04,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:16:05,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:16:05,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:16:10,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:16:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:16:20,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:16:26,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:16:28,798 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:16:35,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:16:35,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:16:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:16:50,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:16:58,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:17:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:17:06,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:17:21,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:17:21,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:17:28,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:17:36,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:17:36,557 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:17:51,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:17:51,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:17:59,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:18:07,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:18:07,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:18:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:18:22,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:18:29,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:18:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:18:37,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:18:52,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:18:52,720 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:18:59,189 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:19:07,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:19:07,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:19:23,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:19:23,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:19:29,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:19:38,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:19:38,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:19:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:19:53,637 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:19:59,342 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:20:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:20:08,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:20:18,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:20,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:20,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:21,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:22,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:24,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:20:24,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:20:25,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:26,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:27,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:27,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:28,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:29,419 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:20:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:30,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:31,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:31,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:32,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:33,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:33,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:34,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:36,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:36,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:37,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:38,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:39,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:20:39,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:20:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:40,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:41,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:42,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:44,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:45,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:45,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:46,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:47,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:47,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:48,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:49,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:50,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:51,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:51,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:52,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:54,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:20:54,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:20:54,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:55,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:20:56,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:58,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:20:59,493 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:21:00,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:02,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:04,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:21:04,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:06,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:08,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:09,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:21:09,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:21:16,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:22,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:25,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:21:25,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:21:26,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:28,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:29,569 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:21:32,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:36,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:39,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:40,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:21:40,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:21:43,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:45,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:49,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:51,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:55,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:21:55,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:21:55,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:57,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:21:59,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:22:01,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:03,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:05,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:07,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:08,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:08,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:08,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:08,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:09,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:22:10,881 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:22:11,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:12,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:12,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:13,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:14,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:15,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:16,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:16,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:17,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:19,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:21,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:23,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:23,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:25,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:25,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:25,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:22:26,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:22:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:27,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:28,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:29,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:22:29,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:31,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:32,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:33,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:34,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:35,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:36,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:36,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:37,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:37,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:39,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:41,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:22:41,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:22:41,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:41,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:42,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:43,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:45,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:46,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:47,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:47,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:49,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:50,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:51,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:53,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:55,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:22:56,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:22:57,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:59,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:22:59,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:22:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:22:59,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:23:01,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:11,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:23:12,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:23:15,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:19,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:23,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:23:27,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:23:27,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:29,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:23:31,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:35,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:37,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:41,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:23:42,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:23:43,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:47,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:49,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:53,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:55,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:23:57,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:23:58,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:23:59,933 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:24:00,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:03,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:03,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:04,755 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 17:24:04,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:04,982 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 17:24:05,086 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 17:24:05,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 17:24:06,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:08,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:09,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:10,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:12,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:13,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:24:13,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:24:14,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:15,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:18,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:20,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:22,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:24,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:25,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:25,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:26,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:27,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:27,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:24:28,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:24:28,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:28,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:30,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:24:30,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:31,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:32,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:33,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:33,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:34,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:35,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:36,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:37,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:38,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:39,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:39,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:40,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:41,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:42,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:43,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:24:43,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:24:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:44,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:45,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:46,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:49,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:49,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:50,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:51,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:51,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:24:53,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:55,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:57,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:24:58,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:24:58,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:25:00,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:25:05,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:05,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:05,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:11,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:14,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:25:14,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:25:15,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:19,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:21,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:25,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:29,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:25:29,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:25:30,194 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:25:31,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:35,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:37,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:41,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:43,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:44,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:25:44,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:25:47,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:49,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:51,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:53,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:56,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:56,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:25:57,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:59,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:25:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:25:59,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:26:00,259 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:26:01,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:03,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:05,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:06,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:06,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:07,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:09,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:11,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:12,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:13,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:13,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:15,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:15,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:26:15,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:26:18,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:18,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:18,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:20,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:22,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:24,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:24,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:25,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:26,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:29,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:30,331 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:26:30,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:26:30,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:26:30,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:30,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:32,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:33,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:33,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:34,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:34,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:36,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:36,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:38,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:38,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:39,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:40,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:42,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:42,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:42,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:44,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:45,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:26:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:26:45,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:26:46,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:48,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:26:56,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:00,407 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:27:00,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:27:00,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:27:02,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:03,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:03,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:06,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:10,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:14,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:16,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:27:16,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:27:18,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:20,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:24,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:28,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:30,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:30,481 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:27:31,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:27:31,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:27:34,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:36,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:38,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:42,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:44,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:46,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:46,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:27:46,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:27:49,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:49,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:27:50,047 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 17:27:50,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:50,268 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 17:27:50,338 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 17:27:51,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 17:27:51,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:52,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:54,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:56,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:27:58,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:00,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:00,563 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:28:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:00,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:01,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:02,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:28:02,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:28:04,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:06,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:07,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:08,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:10,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:12,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:13,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:13,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:14,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:17,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:28:17,421 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:28:18,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:18,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:21,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:22,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:23,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:25,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:26,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:27,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:28,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:29,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,634 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:28:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:31,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:34,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:35,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:37,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:37,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:28:37,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:28:38,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:38,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:39,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:41,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:43,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:50,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:28:52,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:28:52,575 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:28:53,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:28:59,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:00,706 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:29:03,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:06,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:07,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:07,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:29:07,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:29:11,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:15,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:17,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:21,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:29:23,154 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:29:25,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:27,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:30,781 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:29:31,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:33,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:37,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:29:38,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:29:39,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:41,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:45,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:47,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:29:49,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:51,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:53,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:29:53,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:29:57,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:29:59,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:00,851 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:30:01,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:03,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:03,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:06,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:08,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:30:08,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:30:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:10,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:12,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:14,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:14,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:16,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:17,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:18,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:19,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:20,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:21,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:21,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:22,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:30:24,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:30:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:24,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:27,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:28,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:29,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:30,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:30,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:30:31,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:31,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:32,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:32,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:33,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:34,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:34,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:36,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:38,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:38,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:38,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:39,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:30:39,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:30:40,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:42,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:48,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:54,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:30:54,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:30:58,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:30:59,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:30:59,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:01,000 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:31:02,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:04,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:08,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:08,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:09,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:31:09,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:31:12,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:16,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:18,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:22,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:24,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:25,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:31:25,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:31:28,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:30,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:31,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:31:32,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:37,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:39,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:40,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:31:40,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:31:40,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:40,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:40,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:40,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:31:41,701 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 17:31:41,915 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 17:31:41,978 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 17:31:42,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 17:31:42,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:43,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:45,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:49,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:51,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:53,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:55,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:55,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:31:55,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:31:57,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:31:59,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:01,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:32:01,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:03,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:03,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:03,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:04,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:05,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:05,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:07,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:09,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:10,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:32:10,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:32:11,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:11,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:15,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:16,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:17,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:17,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:18,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:19,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:20,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:21,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:21,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:23,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:23,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:24,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:25,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:32:26,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:32:26,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:26,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:27,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:27,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:29,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:29,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:31,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:32:31,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:33,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:41,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:32:41,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:32:43,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:43,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:43,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:45,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:32:47,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:53,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:57,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:32:57,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:32:57,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:32:59,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:01,323 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:33:03,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:04,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:07,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:08,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:08,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:11,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:33:12,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:33:13,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:17,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:19,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:23,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:26,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:33:27,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:33:30,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:31,391 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:33:32,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:34,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:33:38,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:40,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:42,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:43,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:33:43,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:33:44,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:46,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:50,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:52,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:54,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:56,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:33:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:33:58,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:33:58,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:00,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:01,461 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:34:02,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:03,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:04,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:05,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:05,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:06,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:07,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:08,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:10,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:12,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:12,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:34:13,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:34:13,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:13,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:14,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:15,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:16,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:16,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:17,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:18,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:19,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:20,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:21,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:22,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:23,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:23,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:25,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:26,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:27,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:28,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:28,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:34:28,740 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:34:30,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:31,546 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:34:33,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:33,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:40,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:42,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:44,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:34:44,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:34:44,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:50,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:51,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:51,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:54,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:55,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:55,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:34:56,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:34:59,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:34:59,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:35:00,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:01,625 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:35:04,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:07,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:07,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:08,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:10,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:14,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:35:14,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:35:15,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:15,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:17,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:21,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:23,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:27,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:29,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:35:30,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:35:31,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:31,700 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:35:33,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:33,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:33,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:33,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:35:34,818 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 17:35:35,013 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 17:35:35,079 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 17:35:35,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 17:35:35,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:37,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:39,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:41,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:43,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:35:45,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:35:45,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:47,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:49,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:53,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:55,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:57,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:35:59,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:00,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:36:00,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:36:01,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:01,793 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:36:03,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:04,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:04,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:05,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:07,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:07,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:08,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:09,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:11,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:12,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:13,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:14,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:36:15,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:36:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:16,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:17,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:18,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:19,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:20,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:21,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:22,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:23,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:25,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:27,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:28,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:36:31,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:36:31,867 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:36:33,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:33,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:35,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:41,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:46,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:46,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:36:46,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:36:48,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:48,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:48,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:52,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:36:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:36:56,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:00,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:01,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:37:01,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:37:02,151 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:37:02,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:06,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:07,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:08,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:12,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:14,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:17,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:37:17,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:37:17,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:18,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:20,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:22,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:26,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:28,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:28,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:28,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:28,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:28,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:37:30,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:32,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:37:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:37:32,388 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:37:32,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:36,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:38,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:40,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:42,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:44,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:46,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:47,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:37:47,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:37:48,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:50,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:52,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:54,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:56,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:37:58,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:00,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:02,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:38:02,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:02,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:02,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:38:02,851 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:38:04,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:05,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:06,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:08,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:09,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:09,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:10,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:11,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:11,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:12,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:13,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:13,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:14,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:14,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:16,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:17,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:38:18,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:38:18,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:18,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:18,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:20,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:22,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:23,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:23,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:28,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:30,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:31,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:31,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:32,174 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:38:33,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:38:33,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:38:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:36,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:40,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:42,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:44,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:45,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:48,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:48,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:38:48,617 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:38:51,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:51,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:52,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:53,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:55,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:38:55,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:55,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:38:59,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:00,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:00,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:02,250 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:39:03,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:03,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:39:03,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:39:05,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:09,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:11,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:14,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:15,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:17,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:19,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:39:19,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:39:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:19,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:23,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:25,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 17:39:26,348 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 17:39:26,510 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 17:39:26,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 17:39:26,571 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 17:39:27,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:28,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:29,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:31,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:32,588 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:39:33,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:34,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:39:34,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:39:35,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:37,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:39,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:41,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:43,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:45,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:47,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:49,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:39:49,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:39:51,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:53,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:55,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:57,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:39:59,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:01,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:02,663 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:40:03,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:04,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:40:04,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:40:05,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:07,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:09,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:11,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:13,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:15,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:17,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:19,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:20,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:40:20,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:40:21,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:23,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:25,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:27,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:29,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:32,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:32,740 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:40:34,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:40:35,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:40:36,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:38,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:40,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:42,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:44,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:46,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:48,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:50,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:50,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:40:50,772 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:40:52,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:54,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:56,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:40:58,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:00,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:02,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:02,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:41:04,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:41:06,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:41:06,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:08,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:10,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:12,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:14,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:16,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:18,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:20,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:21,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:41:21,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:41:22,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:24,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:26,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:28,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:30,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:32,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:32,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:41:34,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:36,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:41:36,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:41:36,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:38,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:40,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:42,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:44,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:46,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:48,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:51,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:51,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:41:51,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:41:53,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:55,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:57,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:41:59,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:01,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:02,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:42:03,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:05,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:07,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:42:07,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:42:07,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:09,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:11,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:13,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:15,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:17,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:19,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:21,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:42:22,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:42:23,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:25,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:27,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:29,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:31,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:33,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:42:33,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:35,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:37,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:37,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:42:37,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:42:39,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:41,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:43,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:45,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:47,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:49,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:51,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:53,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:42:53,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:42:53,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:55,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:57,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:42:59,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:01,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:03,117 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:43:03,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:05,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:07,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:08,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:43:08,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:43:09,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:11,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:13,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:15,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:17,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:20,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:22,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:23,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:43:23,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:43:24,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:26,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:28,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:30,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:32,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:33,193 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:43:34,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:36,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:38,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:38,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:43:38,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:43:40,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:42,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:44,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:46,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:48,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:50,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:52,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:43:54,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:43:54,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:56,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:43:58,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:00,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:02,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:03,269 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:44:04,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:06,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:08,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:09,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:44:09,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:44:10,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:12,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:14,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:16,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:18,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:20,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:22,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:24,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:24,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:44:24,886 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:44:26,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:28,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:30,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:32,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:33,344 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:44:34,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:36,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:38,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:44:40,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:44:41,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:43,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:45,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:47,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:49,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:51,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:53,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:55,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:44:55,387 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:44:55,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:57,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:44:59,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:01,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:03,417 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:45:03,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:05,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:07,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:09,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:10,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:45:10,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:45:11,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:13,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:15,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:17,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:19,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:21,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:23,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:25,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:25,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:45:25,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:45:27,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:29,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:31,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:33,494 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:45:33,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:35,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:37,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:39,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:41,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:45:41,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:45:41,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:43,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:45,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:47,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:49,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:51,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:53,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:55,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:56,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:45:56,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:45:57,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:45:59,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:01,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:03,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:46:03,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:05,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:07,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:09,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:11,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:11,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:46:11,754 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:46:13,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:15,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:17,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:19,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:21,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:23,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:26,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:27,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:46:27,022 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:46:28,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:30,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:32,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:33,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:46:34,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:36,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:38,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:40,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:42,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:46:42,250 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:46:42,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:44,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:46,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:48,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:50,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:52,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:54,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:56,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:46:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:46:57,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:46:58,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:00,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:02,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:03,720 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:47:04,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:06,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:08,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:10,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:12,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:12,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:47:12,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:47:14,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:16,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:18,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:20,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:22,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:24,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:26,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:28,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:47:28,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:47:28,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:30,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:32,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:33,791 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:47:34,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:37,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:39,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:41,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:43,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:47:43,422 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:47:43,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:45,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:47,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:49,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:51,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:53,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:55,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:57,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:47:58,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:47:58,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:47:59,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:01,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:03,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:03,868 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:48:05,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:07,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:09,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:11,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:13,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:13,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:48:13,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:48:15,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:17,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:19,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:21,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:23,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:25,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:48:29,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:48:29,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:31,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:33,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:33,940 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:48:35,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:37,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:39,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:41,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:43,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:44,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:48:44,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:48:45,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:47,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:49,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:51,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:53,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:55,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:57,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:59,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:48:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:48:59,869 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:49:01,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:03,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:04,015 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:49:05,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:07,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:09,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:11,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:13,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:15,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:49:15,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:49:15,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:17,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:19,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:22,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:24,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:26,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:28,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:30,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:49:30,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:49:30,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:32,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:34,089 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:49:34,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:36,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:38,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:40,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:42,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:44,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:49:45,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:49:46,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:48,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:50,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:52,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:54,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:56,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:49:58,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:00,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:00,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:50:00,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:50:02,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:04,153 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:50:04,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:06,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:08,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:10,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:12,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:14,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:16,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:50:16,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:50:16,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:18,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:20,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:22,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:24,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:26,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:28,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:30,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:31,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:50:31,528 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:50:32,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:34,229 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:50:34,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:36,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:38,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:40,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:42,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:44,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:46,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:50:46,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:50:46,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:48,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:50,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:53,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:55,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:57,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:50:59,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:01,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:01,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:51:01,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:51:03,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:04,307 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:51:05,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:07,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:09,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:11,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:13,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:15,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:17,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:51:17,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:51:17,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:19,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:21,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:23,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:25,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:27,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:29,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:31,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:51:32,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:51:33,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:34,382 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:51:35,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:37,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:39,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:41,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:43,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:45,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:51:47,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:51:47,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:49,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:51,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:53,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:55,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:51:58,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:00,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:02,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:03,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:52:03,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:52:04,459 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:52:04,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:06,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:08,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:10,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:12,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:14,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:16,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:18,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:52:18,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:52:18,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:20,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:22,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:24,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:26,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:28,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:30,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:32,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:33,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:52:33,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:52:34,534 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:52:34,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:36,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:38,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:40,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:42,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:44,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:46,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:48,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:52:48,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:52:48,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:50,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:52,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:54,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:56,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:52:58,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:00,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:02,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:53:04,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:53:04,610 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:53:04,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:06,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:08,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:10,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:12,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:14,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:16,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:18,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:53:19,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:53:20,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:22,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:24,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:27,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:29,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:31,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:33,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:34,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:53:34,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:53:34,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:53:35,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:37,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:39,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:41,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:43,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:45,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:47,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:49,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:49,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:53:49,963 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:53:51,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:53,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:55,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:57,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:53:59,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:01,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:03,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:04,761 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:54:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:54:05,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:54:05,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:07,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:09,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:11,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:13,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:15,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:17,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:19,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:54:20,542 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:54:21,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:23,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:25,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:27,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:29,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:31,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:33,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:34,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:54:35,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:54:35,806 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:54:35,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:37,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:39,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:41,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:43,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:45,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:47,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:49,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:51,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:54:51,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:54:51,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:53,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:55,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:57,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:54:59,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:02,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:04,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:55:04,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:06,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:55:06,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:55:06,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:08,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:10,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:12,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:14,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:16,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:18,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:20,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:21,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:55:21,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:55:22,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:24,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:26,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:28,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:30,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:32,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:34,983 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:55:34,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:36,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:55:37,040 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:55:38,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:40,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:42,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:44,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:46,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:48,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:50,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:55:52,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:55:52,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:54,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:56,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:55:58,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:00,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:03,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:05,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:05,057 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:56:07,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:07,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:56:07,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:56:09,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:11,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:13,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:15,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:17,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:19,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:21,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:56:22,851 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:56:23,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:25,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:27,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:29,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:31,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:33,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:35,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:35,133 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:56:37,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:38,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:56:38,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:56:39,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:41,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:43,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:45,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:47,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:49,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:51,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:53,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:56:53,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:56:54,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:56,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:56:58,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:00,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:02,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:04,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:05,208 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:57:06,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:08,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:08,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:57:08,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:57:10,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:12,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:14,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:16,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:18,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:20,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:22,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:24,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:57:24,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:57:24,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:26,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:28,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:30,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:32,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:34,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:35,284 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:57:36,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:38,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:57:39,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:57:40,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:42,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:44,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:46,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:48,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:50,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:52,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:54,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:54,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:57:54,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:57:56,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:57:58,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:00,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:02,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:04,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:05,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:58:06,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:08,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:58:09,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:58:10,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:12,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:14,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:16,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:18,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:20,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:22,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:25,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:25,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:58:25,422 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:58:27,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:29,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:31,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:33,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:35,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:35,433 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:58:37,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:39,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:58:40,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:58:41,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:43,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:45,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:47,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:49,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:51,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:53,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:55,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:58:55,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:58:57,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:58:59,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:01,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:03,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:05,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:05,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:59:07,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:09,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:11,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:11,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:59:11,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:59:13,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:15,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:17,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:19,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:21,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:23,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:25,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:59:26,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:59:27,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:29,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:31,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:33,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:35,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:35,584 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 17:59:37,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:39,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:41,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:41,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:59:41,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:59:43,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:45,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:48,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:50,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:52,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:54,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:56,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 17:59:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 17:59:57,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 17:59:58,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:00,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:02,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:04,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:05,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:00:06,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:08,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:10,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:12,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:12,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:00:12,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:00:14,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:16,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:18,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:20,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:22,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:24,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:26,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:27,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:00:27,579 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:00:28,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:30,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:32,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:34,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:35,733 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:00:36,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:38,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:40,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:42,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:42,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:00:42,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:00:44,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:46,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:48,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:50,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:52,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:54,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:56,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:58,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:00:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:00:58,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:01:00,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:02,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:04,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:05,809 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:01:06,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:08,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:10,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:12,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:01:13,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:01:14,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:16,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:18,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:20,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:22,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:24,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:27,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:01:28,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:01:29,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:31,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:33,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:35,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:35,883 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:01:37,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:39,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:41,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:43,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:01:44,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:01:45,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:47,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:49,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:51,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:53,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:55,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:57,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:59,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:01:59,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:01:59,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:02:01,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:03,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:05,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:05,951 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:02:07,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:09,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:11,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:13,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:14,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:02:14,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:02:15,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:17,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:19,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:21,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:23,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:25,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:27,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:29,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:30,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:02:30,099 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:02:31,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:33,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:35,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:36,022 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:02:37,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:39,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:41,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:43,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:45,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:02:45,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:02:47,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:49,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:51,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:53,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:55,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:02:57,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:00,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:03:00,589 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:03:02,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:04,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:06,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:03:06,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:08,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:10,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:12,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:14,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:03:15,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:03:16,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:18,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:20,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:22,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:24,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:26,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:28,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:30,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:31,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:03:31,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:03:32,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:34,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:36,163 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:03:36,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:38,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:40,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:42,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:44,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:46,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:46,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:03:46,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:03:48,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:50,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:52,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:54,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:56,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:03:58,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:00,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:01,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:04:01,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:04:02,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:04,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:06,240 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:04:07,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:09,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:11,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:13,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:15,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:17,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:04:17,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:04:17,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:19,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:21,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:23,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:25,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:27,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:29,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:31,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:32,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:04:32,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:04:33,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:35,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:36,315 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:04:37,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:39,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:41,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:43,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:45,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:47,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:47,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:04:47,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:04:49,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:51,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:53,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:55,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:57,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:04:59,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:01,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:05:02,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:05:03,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:05,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:06,384 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:05:07,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:09,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:11,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:13,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:15,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:17,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:18,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:05:18,156 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:05:19,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:22,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:24,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:26,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:28,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:30,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:32,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:33,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:05:33,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:05:34,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:36,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:36,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:05:38,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:40,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:42,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:44,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:46,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:48,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:48,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:05:48,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:05:50,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:52,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:54,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:56,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:05:58,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:00,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:02,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:03,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:06:03,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:06:04,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:06,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:06,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:06:08,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:10,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:12,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:14,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:16,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:18,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:06:19,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:06:20,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:22,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:24,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:26,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:28,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:30,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:32,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:34,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:34,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:06:34,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:06:36,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:36,572 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:06:38,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:40,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:42,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:44,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:46,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:48,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:49,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:06:49,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:06:50,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:53,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:55,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:57,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:06:59,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:01,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:03,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:05,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:07:05,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:07:05,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:06,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:07:07,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:09,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:11,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:13,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:15,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:17,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:19,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:07:20,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:07:21,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:23,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:25,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:27,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:29,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:31,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:33,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:35,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:35,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:07:35,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:07:36,710 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:07:37,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:39,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:41,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:43,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:45,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:47,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:49,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:07:50,978 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:07:51,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:53,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:55,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:57,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:07:59,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:01,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:03,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:05,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:08:06,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:08:06,783 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:08:07,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:09,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:11,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:13,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:15,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:17,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:19,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:21,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:21,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:08:21,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:08:23,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:25,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:27,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:29,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:31,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:33,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:35,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:36,858 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:08:36,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:08:36,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:08:37,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:39,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:41,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:43,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:45,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:47,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:49,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:51,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:52,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:08:52,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:08:54,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:56,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:08:58,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:00,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:02,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:04,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:06,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:06,926 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:09:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:09:07,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:09:08,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:10,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:12,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:14,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:16,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:18,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:20,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:22,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:22,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:09:22,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:09:24,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:26,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:28,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:30,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:32,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:34,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:36,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:36,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:09:38,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:09:38,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:09:40,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:42,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:44,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:46,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:48,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:50,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:52,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:09:53,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:09:54,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:56,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:09:58,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:00,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:02,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:04,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:06,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:07,050 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:10:08,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:09,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:10:09,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:10:10,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:12,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:14,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:16,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:18,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:20,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:22,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:24,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:24,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:10:24,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:10:26,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:28,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:30,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:32,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:34,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:36,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:37,125 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:10:38,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:39,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:10:39,863 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:10:40,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:42,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:44,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:46,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:48,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:51,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:53,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:55,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:10:55,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:10:55,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:57,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:10:59,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:01,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:03,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:05,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:07,200 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:11:07,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:09,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:10,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:11:10,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:11:11,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:13,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:15,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:17,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:19,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:21,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:23,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:25,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:26,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:11:26,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:11:27,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:29,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:31,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:33,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:35,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:37,276 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:11:37,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:39,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:41,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:11:41,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:11:41,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:43,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:45,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:47,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:49,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:51,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:53,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:55,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:56,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:11:56,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:11:57,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:11:59,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:01,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:03,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:05,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:07,352 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:12:07,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:09,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:11,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:11,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:12:11,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:12:13,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:15,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:17,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:19,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:21,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:23,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:26,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:27,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:12:27,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:12:28,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:30,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:32,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:34,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:36,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:37,423 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:12:38,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:40,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:12:42,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:12:42,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:44,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:46,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:48,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:50,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:52,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:54,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:56,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:12:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:12:57,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:12:58,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:00,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:02,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:04,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:06,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:07,478 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:13:08,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:10,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:12,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:12,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:13:12,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:13:14,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:16,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:18,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:20,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:22,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:24,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:26,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:28,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:13:28,238 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:13:28,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:30,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:32,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:34,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:36,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:37,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:13:38,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:40,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:42,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:43,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:13:43,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:13:44,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:46,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:48,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:50,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:52,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:54,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:56,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:58,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:13:58,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:13:58,852 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:14:00,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:02,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:04,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:06,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:07,629 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:14:08,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:10,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:12,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:14,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:15,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:14:15,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:14:16,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:18,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:20,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:22,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:25,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:27,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:29,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:14:31,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:14:31,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:33,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:35,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:37,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:37,706 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:14:39,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:41,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:43,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:45,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:46,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:14:46,403 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:14:47,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:49,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:51,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:53,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:55,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:57,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:14:59,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:01,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:15:01,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:15:03,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:05,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:07,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:07,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:15:09,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:11,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:13,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:15,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:15:16,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:15:17,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:19,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:21,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:23,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:25,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:27,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:30,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:32,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:15:32,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:15:32,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:34,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:36,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:37,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:15:38,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:40,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:42,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:44,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:46,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:47,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:15:47,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:15:48,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:50,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:52,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:54,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:56,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:15:58,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:00,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:02,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:02,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:16:02,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:16:04,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:06,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:07,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:16:08,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:10,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:12,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:14,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:16,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:18,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:16:18,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:16:18,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:20,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:22,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:24,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:26,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:28,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:30,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:32,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:16:33,338 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:16:34,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:37,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:37,985 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:16:39,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:41,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:43,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:45,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:47,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:48,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:16:48,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:16:49,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:51,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:53,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:55,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:57,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:16:59,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:01,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:03,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:03,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:17:03,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:17:05,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:07,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:08,060 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:17:09,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:11,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:13,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:15,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:17,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:19,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:17:19,149 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:17:19,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:21,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:23,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:25,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:28,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:30,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:32,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:34,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:17:34,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:17:34,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:36,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:38,136 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:17:38,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:40,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:42,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:44,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:46,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:48,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:49,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:17:49,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:17:50,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:52,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:54,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:56,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:17:58,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:00,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:02,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:04,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:18:05,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:18:06,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:08,204 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:18:08,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:10,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:12,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:14,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:16,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:18,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:20,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:18:20,329 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:18:20,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:22,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:24,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:26,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:28,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:30,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:32,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:34,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:35,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:18:35,653 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:18:36,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:38,272 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:18:38,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:40,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:42,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:44,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:46,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:48,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:50,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:50,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:18:50,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:18:52,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:54,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:56,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:18:58,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:00,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:02,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:04,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:06,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:19:06,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:19:06,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:08,345 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:19:08,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:11,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:13,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:15,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:17,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:19,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:21,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:19:21,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:19:21,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:23,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:25,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:27,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:29,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:31,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:33,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:35,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:36,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:19:36,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:19:37,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:38,415 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:19:39,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:41,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:43,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:45,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:47,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:49,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:51,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:19:52,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:19:53,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:55,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:57,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:19:59,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:01,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:03,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:05,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:07,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:20:07,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:20:07,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:08,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:20:09,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:11,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:13,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:15,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:17,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:19,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:21,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:22,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:20:22,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:20:23,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:25,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:27,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:29,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:31,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:33,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:35,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:37,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:20:37,875 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:20:38,553 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:20:38,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:40,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:42,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:44,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:46,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:48,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:50,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:52,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:53,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:20:53,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:20:54,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:56,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:20:58,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:00,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:02,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:04,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:06,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:21:08,406 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:21:08,636 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:21:08,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:10,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:12,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:14,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:16,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:18,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:20,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:22,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:23,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:21:23,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:21:24,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:26,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:28,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:21:38,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:21:38,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:21:38,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:21:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:21:54,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:22:08,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:22:09,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:22:09,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:22:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:22:24,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:22:29,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:22:29,676 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 18:22:29,715 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 18:22:29,790 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 18:22:30,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 18:22:30,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:22:31,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:22:36,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:22:38,859 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:22:39,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:22:39,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:22:53,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:22:55,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:22:55,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:23:08,934 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:23:10,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:23:10,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:23:25,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:23:25,658 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:23:39,008 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:23:41,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:23:41,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:23:56,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:23:56,343 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:24:09,085 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:24:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:24:11,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:24:26,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:24:26,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:24:39,164 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:24:42,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:24:42,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:24:57,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:24:57,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:25:09,245 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:25:12,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:25:12,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:25:27,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:25:27,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:25:39,323 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:25:43,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:25:43,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:25:58,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:25:58,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:26:09,400 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:26:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:26:13,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:26:28,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:26:28,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:26:39,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:26:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:26:44,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:26:45,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:26:49,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:26:51,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:26:53,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:26:55,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:26:57,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:26:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:26:59,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:26:59,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:01,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:03,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:05,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:07,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:09,550 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:27:09,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:11,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:13,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:14,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:27:14,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:27:15,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:17,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:17,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:19,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:20,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:20,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:21,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:23,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:23,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:25,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:27,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:27,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:28,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:29,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:27:29,907 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:27:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:31,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:32,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:33,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:34,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:34,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:35,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:37,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:38,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:39,616 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:27:44,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:44,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:45,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:27:45,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:27:46,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:47,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:49,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:51,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:52,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:53,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:53,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:54,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:27:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:27:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:28:00,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:28:00,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:00,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:04,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:04,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:05,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:08,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:09,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:28:12,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:14,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:15,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:28:15,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:28:17,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:18,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:20,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:24,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:25,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:25,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:26,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:30,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:28:30,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:28:30,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:32,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:34,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:36,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:38,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:39,761 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:28:40,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:40,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:40,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:40,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:40,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:41,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:42,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:45,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:28:46,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:28:46,057 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:28:46,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:48,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:50,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:52,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:54,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:56,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:28:58,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:00,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:01,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:29:01,282 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:29:02,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:04,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:06,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:08,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:09,837 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:29:10,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:12,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:14,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:16,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:29:16,602 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:29:16,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:17,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:18,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:20,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:22,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:22,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:24,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:26,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:26,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:28,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:28,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:29,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:30,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:31,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:29:31,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:29:31,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:32,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:34,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:39,913 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:29:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:42,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:45,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:45,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:46,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:29:47,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:29:47,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:47,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:49,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:50,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:52,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:54,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:56,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:56,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:56,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:56,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:56,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:56,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:57,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:29:58,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:58,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:29:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:00,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:02,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:30:02,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:30:02,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:03,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:06,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:07,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:09,990 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:30:10,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:13,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:16,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:17,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:30:17,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:30:17,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:18,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:18,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:19,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:23,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:25,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:25,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:29,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:31,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:32,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:30:32,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:30:33,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:35,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:36,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:36,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:36,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:36,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:37,298 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 18:30:37,515 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 18:30:37,577 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 18:30:37,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 18:30:38,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:39,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:40,089 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:30:40,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:40,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:41,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:43,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:45,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:30:47,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:30:48,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:30:49,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:51,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:53,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:55,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:57,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:30:59,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:01,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:03,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:31:03,292 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:31:03,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:05,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:07,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:09,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:10,164 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:31:11,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:13,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:15,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:15,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:17,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:18,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:31:18,579 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:31:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:19,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:21,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:23,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:23,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:25,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:25,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:26,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:26,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:27,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:29,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:33,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:31:33,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:31:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:34,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:37,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:40,238 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:31:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:40,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:41,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:45,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:48,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:49,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:31:49,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:31:49,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:50,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:50,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:53,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:55,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:55,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:57,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:31:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:58,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:31:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:00,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:01,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:03,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:04,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:04,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:32:04,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:32:07,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:07,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:10,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:32:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:10,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:13,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:14,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:16,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:17,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:17,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:32:19,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:32:20,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:22,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:24,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:26,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:30,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:32,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:34,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:32:35,056 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:32:36,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:39,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:39,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,376 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:32:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:32:40,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:42,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:44,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:46,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:48,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:50,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:32:50,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:32:50,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:52,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:54,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:56,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:32:58,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:00,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:03,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:05,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:05,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:33:05,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:33:07,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:09,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:09,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:10,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:33:11,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:13,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:15,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:15,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:17,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:17,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:19,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:19,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:20,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:33:20,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:33:21,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:23,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:27,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:31,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:31,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:35,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:36,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:33:36,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:33:37,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:38,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:40,529 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:33:41,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:42,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:42,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:44,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:45,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:48,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:48,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:49,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:50,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:33:51,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:33:52,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:52,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:53,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:53,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:55,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:33:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:57,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:58,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:33:59,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:00,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:03,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:03,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:04,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:04,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:04,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:04,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:04,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:05,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:06,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:06,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:34:06,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:34:08,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:08,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:09,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:10,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:34:11,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:15,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:15,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:15,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:17,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:17,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:20,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:20,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:34:22,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:34:22,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:22,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:24,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:25,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:25,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:25,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:26,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:26,475 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 18:34:26,688 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 18:34:26,775 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 18:34:27,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 18:34:28,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:30,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:34,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:36,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:34:37,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:34:38,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:38,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:38,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:40,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:40,811 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:34:41,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:34:42,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:44,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:46,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:48,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:50,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:52,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:52,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:34:52,461 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:34:54,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:56,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:34:58,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:00,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:02,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:04,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:06,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:07,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:35:07,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:35:08,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:08,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:10,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:10,881 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:35:12,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:14,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:16,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:16,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:16,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:18,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:20,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:35:22,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:35:28,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:28,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:31,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:33,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:34,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:35,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:35,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:38,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:38,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:35:38,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:35:40,956 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:35:41,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:41,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:42,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:45,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:46,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:48,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:49,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:50,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:50,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:52,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:52,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:35:53,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:35:53,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:53,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:56,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:35:57,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:35:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:00,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:01,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:02,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:03,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:06,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:07,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:07,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:08,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:08,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:36:08,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:36:10,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:10,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:10,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,034 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:36:11,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:11,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:13,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:14,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:16,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:18,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:19,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:21,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:21,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:23,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:23,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:36:23,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:36:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:25,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:27,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:28,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:28,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:29,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:30,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:30,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:31,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:35,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:37,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:39,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:39,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:36:39,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:36:41,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:41,111 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:36:41,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:41,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:43,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:44,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:36:45,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:47,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:49,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:51,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:53,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:54,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:36:54,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:36:55,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:57,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:36:59,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:01,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:03,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:05,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:07,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:08,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:08,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:09,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:37:09,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:37:11,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:11,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:37:13,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:13,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:13,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:15,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:23,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:25,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:37:25,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:37:29,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:33,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:34,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:34,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:37,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:37,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:39,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:37:40,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:37:40,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:40,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:41,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:37:41,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:43,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:45,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:45,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:46,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:47,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:50,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:51,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:53,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:53,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:53,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:55,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:37:55,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:37:56,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:58,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:37:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:37:59,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:00,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:02,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:03,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:04,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:05,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:05,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:06,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:07,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:07,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:09,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:10,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:10,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:11,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:38:11,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:38:11,356 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:38:12,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:12,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:12,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:14,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:14,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:16,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:16,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:16,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:16,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:17,612 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 18:38:17,818 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 18:38:17,884 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 18:38:18,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 18:38:18,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:19,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:19,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:20,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:22,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:22,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:24,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:24,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:26,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:26,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:38:26,447 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:38:26,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:26,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:28,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:30,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:34,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:36,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:38,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:39,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:39,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:40,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:41,442 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:38:41,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:38:41,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:38:42,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:44,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:38:44,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:46,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:48,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:50,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:52,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:54,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:56,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:38:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:38:56,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:38:58,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:00,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:00,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:00,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:02,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:04,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:04,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:04,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:06,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:08,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:10,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:11,520 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:39:12,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:39:12,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:39:20,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:24,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:27,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:39:27,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:39:28,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:30,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:32,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:36,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:36,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:36,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:38,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:40,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:40,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:41,596 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:39:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:39:42,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:39:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:43,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:44,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:45,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:49,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:50,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:51,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:52,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:52,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:53,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:54,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:57,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:39:58,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:39:58,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:39:59,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:39:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:03,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:04,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:05,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:05,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:06,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:07,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:08,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:08,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:09,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:11,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:11,673 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:40:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:40:13,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:40:13,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:13,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:16,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:17,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:21,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:21,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:23,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:25,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:27,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:40:28,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:40:29,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:30,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:31,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:32,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:32,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:33,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:35,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:37,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:38,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:39,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:41,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:41,747 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:40:43,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:43,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:40:43,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:40:45,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:47,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:49,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:51,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:53,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:55,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:56,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:40:57,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:40:59,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:40:59,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:40:59,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:01,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:02,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:02,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:03,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:05,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:11,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:41:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:41:14,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:41:15,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:21,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:25,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:29,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:29,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:41:29,553 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:41:33,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:35,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:35,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:39,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:40,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:41,898 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:41:42,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:42,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:43,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:44,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:41:44,830 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:41:45,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:45,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:49,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:49,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:51,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:55,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:55,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:57,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:41:58,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:41:59,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:42:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:42:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:01,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:03,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:03,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:03,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:05,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:08,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:09,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:09,147 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 18:42:09,360 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 18:42:09,437 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 18:42:10,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 18:42:11,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:12,044 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:42:12,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:13,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:14,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:42:15,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:42:15,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:15,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:17,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:18,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:19,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:21,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:23,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:23,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:25,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:25,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:27,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:28,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:30,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:30,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:42:30,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:42:32,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:32,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:34,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:35,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:35,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:36,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:38,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:38,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:40,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:42,121 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:42:42,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:44,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:42:45,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:42:46,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:48,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:50,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:52,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:52,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:52,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:42:54,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:56,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:42:58,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:00,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:01,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:43:01,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:43:08,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:12,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:43:14,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:15,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:15,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:16,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:43:16,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:43:18,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:22,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:26,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:30,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:31,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:43:31,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:43:32,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:32,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:36,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:37,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:37,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:38,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:38,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:42,273 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:43:42,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:43,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:44,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:45,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:45,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:43:47,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:43:47,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:47,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:48,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:48,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:50,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:51,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:52,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:54,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:56,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:57,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:58,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:43:59,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:43:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:00,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:01,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:01,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:44:02,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:44:02,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:02,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:03,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:05,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:06,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:07,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:08,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:09,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:09,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:10,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:11,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:12,353 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:44:12,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:14,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:16,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:17,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:44:17,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:44:18,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:19,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:20,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:22,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:22,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:25,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:27,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:29,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:29,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:29,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:31,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:32,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:44:32,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:44:33,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:35,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:35,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:36,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:37,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:39,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:41,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:42,433 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:44:43,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:45,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:47,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:48,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:44:48,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:44:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:48,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:44:49,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:51,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:44:53,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:01,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:03,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:45:03,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:45:05,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:09,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:12,506 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:45:13,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:17,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:18,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:45:18,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:45:21,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:23,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:27,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:28,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:31,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:32,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:32,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:33,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:33,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:45:33,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:45:34,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:34,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:37,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:37,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:39,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:39,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:41,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:42,583 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:45:43,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:44,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:45,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:47,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:47,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:48,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:45:48,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:45:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:49,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:50,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:52,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:53,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 18:45:53,262 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 18:45:53,475 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 18:45:53,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:53,532 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 18:45:54,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 18:45:54,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:55,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:57,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:45:59,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:01,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:03,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:46:04,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:46:05,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:07,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:09,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:11,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:12,680 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:46:13,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:15,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:17,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:19,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:46:19,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:46:21,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:23,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:25,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:27,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:29,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:31,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:33,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:34,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:46:34,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:46:35,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:37,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:39,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:41,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:42,752 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:46:43,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:45,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:47,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:49,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:50,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:46:50,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:46:51,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:53,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:55,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:57,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:46:59,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:01,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:03,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:05,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:47:05,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:47:05,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:07,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:09,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:12,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:12,826 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:47:14,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:16,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:18,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:20,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:20,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:47:20,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:47:22,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:24,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:26,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:28,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:30,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:32,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:34,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:36,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:47:36,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:47:36,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:38,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:40,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:42,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:42,902 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:47:44,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:46,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:48,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:50,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:51,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:47:51,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:47:52,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:54,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:56,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:47:58,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:00,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:02,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:04,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:06,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:06,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:48:06,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:48:08,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:10,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:12,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:12,980 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:48:14,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:16,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:18,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:20,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:48:21,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:48:22,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:24,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:26,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:28,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:30,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:32,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:34,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:36,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:48:37,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:48:38,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:40,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:42,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:43,054 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:48:45,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:47,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:49,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:51,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:52,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:48:52,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:48:53,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:55,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:57,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:48:59,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:01,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:03,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:05,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:07,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:49:07,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:49:09,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:11,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:13,136 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:49:13,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:15,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:17,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:19,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:21,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:23,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:49:23,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:49:23,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:25,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:27,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:29,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:31,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:33,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:35,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:37,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:38,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:49:38,268 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:49:39,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:41,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:43,212 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:49:43,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:45,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:47,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:49,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:51,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:53,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:49:53,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:49:53,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:55,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:57,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:49:59,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:01,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:03,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:05,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:07,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:08,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:50:08,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:50:09,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:11,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:13,289 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:50:14,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:16,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:18,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:20,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:22,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:24,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:50:24,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:50:24,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:26,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:28,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:30,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:32,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:34,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:36,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:38,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:50:39,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:50:40,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:42,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:43,362 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:50:44,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:46,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:48,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:50,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:52,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:54,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:50:54,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:50:54,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:56,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:50:58,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:00,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:02,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:04,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:06,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:08,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:51:09,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:51:10,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:12,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:13,430 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:51:14,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:16,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:18,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:20,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:22,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:24,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:25,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:51:25,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:51:26,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:28,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:30,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:32,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:34,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:36,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:38,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:40,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:51:40,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:51:41,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:43,500 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:51:43,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:45,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:47,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:49,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:51,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:53,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:51:55,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:51:55,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:57,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:51:59,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:01,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:03,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:05,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:07,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:09,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:10,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:52:10,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:52:11,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:13,571 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:52:13,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:15,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:17,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:19,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:21,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:23,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:25,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:26,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:52:26,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:52:27,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:29,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:31,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:33,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:35,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:37,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:39,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:41,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:52:41,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:52:41,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:43,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:43,647 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:52:45,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:47,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:49,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:51,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:53,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:55,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:52:56,860 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:52:57,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:52:59,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:01,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:03,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:05,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:07,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:09,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:11,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:12,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:53:12,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:53:13,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:13,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:53:16,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:18,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:20,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:22,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:24,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:26,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:27,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:53:27,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:53:28,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:30,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:32,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:34,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:36,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:38,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:40,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:53:42,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:53:42,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:43,798 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:53:44,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:46,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:48,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:50,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:52,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:54,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:56,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:53:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:53:57,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:53:58,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:00,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:02,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:04,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:06,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:08,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:10,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:12,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:13,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:54:13,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:54:13,872 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:54:14,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:16,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:18,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:20,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:22,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:24,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:26,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:28,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:54:28,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:54:28,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:30,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:33,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:35,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:37,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:39,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:41,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:43,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:54:43,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:54:44,041 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:54:45,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:47,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:49,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:51,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:53,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:55,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:57,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:54:59,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:54:59,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:54:59,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:01,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:03,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:05,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:07,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:09,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:11,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:13,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:14,024 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:55:14,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:55:14,497 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:55:15,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:17,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:19,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:21,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:23,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:25,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:27,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:29,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:29,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:55:29,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:55:31,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:33,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:35,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:37,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:39,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:41,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:43,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:44,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:55:45,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:55:45,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:55:45,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:48,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:50,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:52,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:54,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:56,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:55:58,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:00,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:56:00,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:56:00,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:02,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:04,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:06,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:08,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:10,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:12,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:14,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:56:14,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:56:15,828 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:56:16,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:18,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:20,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:22,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:24,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:26,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:28,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:30,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:31,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:56:31,052 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:56:32,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:34,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:36,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:38,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:40,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:42,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:44,252 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:56:44,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:46,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:56:46,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:56:46,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:48,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:50,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:52,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:54,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:56,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:56:58,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:00,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:01,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:57:01,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:57:02,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:04,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:06,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:08,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:10,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:12,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:14,329 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:57:15,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:16,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:57:16,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:57:17,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:19,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:21,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:23,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:25,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:27,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:29,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:31,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:57:32,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:57:33,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:35,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:37,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:39,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:41,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:43,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:44,404 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:57:45,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:47,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:57:47,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:57:49,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:51,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:53,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:55,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:57,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:57:59,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:01,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:58:03,065 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:58:03,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:05,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:07,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:09,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:11,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:13,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:14,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:58:15,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:17,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:58:18,292 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:58:19,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:22,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:24,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:26,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:28,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:30,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:32,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:33,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:58:33,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:58:34,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:36,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:38,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:40,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:42,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:44,551 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:58:44,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:46,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:48,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:58:48,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:58:48,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:50,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:52,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:54,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:56,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:58:58,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:00,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:02,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:04,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:59:04,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:59:04,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:06,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:08,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:10,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:12,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:14,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:59:14,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:16,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:18,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:19,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:59:19,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:59:20,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:22,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:24,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:26,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:28,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:30,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:32,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:34,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:59:34,560 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:59:34,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:36,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:38,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:40,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:42,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:44,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 18:59:44,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:46,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:48,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:49,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 18:59:49,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 18:59:50,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:53,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:55,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:57,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 18:59:59,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:01,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:03,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:00:05,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:00:05,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:07,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:09,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:11,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:13,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:14,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:00:15,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:17,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:19,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:20,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:00:20,406 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:00:21,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:23,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:25,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:27,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:29,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:31,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:33,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:35,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:00:35,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:00:35,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:37,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:39,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:41,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:43,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:44,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:00:45,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:47,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:49,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:50,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:00:50,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:00:51,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:53,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:55,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:57,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:00:59,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:01,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:03,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:05,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:06,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:01:06,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:01:07,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:09,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:11,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:13,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:14,912 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:01:16,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:18,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:20,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:21,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:01:21,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:01:22,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:24,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:26,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:28,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:30,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:32,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:34,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:36,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:01:36,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:01:36,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:38,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:40,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:42,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:44,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:44,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:01:46,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:48,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:50,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:01:51,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:01:52,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:54,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:56,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:01:58,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:00,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:02,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:04,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:06,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:02:07,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:02:08,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:10,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:12,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:14,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:15,060 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:02:16,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:18,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:20,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:02:22,371 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:02:22,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:24,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:26,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:28,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:30,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:32,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:34,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:36,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:37,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:02:37,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:02:38,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:40,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:42,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:44,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:45,136 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:02:46,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:48,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:50,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:52,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:52,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:02:52,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:02:54,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:56,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:02:58,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:00,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:02,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:04,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:06,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:03:08,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:03:08,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:10,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:12,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:14,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:15,211 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:03:16,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:18,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:20,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:22,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:03:23,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:03:24,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:26,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:28,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:31,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:33,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:35,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:37,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:38,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:03:38,786 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:03:39,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:41,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:43,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:45,286 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:03:45,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:47,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:49,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:51,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:53,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:03:54,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:03:55,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:57,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:03:59,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:01,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:03,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:05,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:07,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:09,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:04:09,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:04:09,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:11,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:13,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:15,361 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:04:15,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:17,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:19,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:21,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:23,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:04:24,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:04:25,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:27,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:29,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:31,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:33,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:35,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:37,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:39,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:04:39,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:04:39,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:41,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:43,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:45,437 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:04:45,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:47,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:50,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:52,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:54,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:55,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:04:55,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:04:56,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:04:58,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:00,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:02,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:04,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:06,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:08,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:10,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:05:10,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:05:10,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:12,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:14,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:15,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:05:16,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:18,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:20,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:22,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:24,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:25,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:05:25,816 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:05:26,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:28,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:30,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:32,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:34,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:36,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:38,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:40,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:41,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:05:41,063 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:05:42,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:44,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:45,586 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:05:46,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:48,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:50,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:52,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:54,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:56,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:05:56,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:05:56,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:05:58,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:00,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:02,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:04,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:06,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:08,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:10,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:06:11,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:06:12,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:14,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:15,665 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:06:16,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:18,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:20,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:22,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:24,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:26,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:06:26,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:06:26,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:28,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:30,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:32,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:34,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:36,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:38,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:40,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:06:42,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:06:43,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:45,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:06:45,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:47,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:49,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:51,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:53,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:55,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:57,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:06:57,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:06:57,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:06:59,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:01,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:03,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:06,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:08,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:10,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:12,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:07:12,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:07:14,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:15,798 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:07:16,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:18,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:20,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:22,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:24,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:26,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:28,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:07:28,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:07:30,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:32,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:34,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:36,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:38,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:40,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:42,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:43,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:07:43,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:07:44,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:45,870 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:07:46,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:48,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:50,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:52,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:54,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:56,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:58,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:07:58,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:07:58,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:08:00,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:02,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:04,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:06,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:08,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:10,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:12,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:14,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:08:14,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:08:15,944 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:08:16,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:18,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:20,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:23,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:25,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:27,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:29,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:29,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:08:29,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:08:31,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:33,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:35,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:37,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:39,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:41,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:43,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:44,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:08:44,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:08:45,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:46,019 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:08:47,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:49,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:51,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:53,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:55,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:57,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:59,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:08:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:08:59,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:09:01,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:03,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:05,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:07,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:09,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:11,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:13,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:15,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:15,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:09:15,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:09:16,094 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:09:17,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:19,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:21,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:23,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:25,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:27,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:29,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:09:30,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:09:31,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:33,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:35,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:37,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:39,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:41,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:43,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:45,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:45,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:09:45,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:09:46,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:09:47,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:49,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:51,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:53,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:55,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:57,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:09:59,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:00,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:10:00,946 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:10:02,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:04,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:06,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:08,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:10,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:12,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:14,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:16,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:16,236 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:10:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:10:16,254 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:10:18,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:20,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:22,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:24,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:26,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:28,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:30,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:31,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:10:31,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:10:32,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:34,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:36,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:38,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:40,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:42,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:44,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:46,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:46,307 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:10:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:10:46,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:10:48,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:50,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:52,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:54,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:56,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:10:58,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:00,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:02,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:02,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:11:02,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:11:04,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:06,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:08,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:10,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:12,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:14,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:16,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:16,368 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:11:17,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:11:17,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:11:18,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:20,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:22,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:25,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:27,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:29,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:31,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:32,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:11:32,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:11:33,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:35,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:37,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:39,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:41,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:43,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:45,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:46,437 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:11:47,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:11:47,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:11:49,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:51,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:53,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:55,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:57,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:11:59,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:01,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:03,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:12:03,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:12:03,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:05,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:07,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:09,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:11,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:13,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:15,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:16,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:12:17,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:18,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:12:18,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:12:19,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:21,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:23,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:25,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:27,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:29,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:31,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:33,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:33,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:12:33,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:12:35,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:37,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:39,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:41,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:43,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:45,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:46,570 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:12:47,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:49,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:12:49,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:12:49,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:51,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:53,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:55,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:12:58,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:00,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:02,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:04,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:04,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:13:04,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:13:06,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:08,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:10,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:12,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:14,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:16,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:16,635 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:13:18,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:19,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:13:19,580 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:13:20,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:22,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:24,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:26,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:28,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:30,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:32,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:34,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:13:34,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:13:36,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:38,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:40,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:42,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:44,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:46,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:46,695 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:13:48,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:50,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:13:50,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:13:50,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:52,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:54,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:56,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:13:58,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:00,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:02,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:04,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:05,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:14:05,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:14:06,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:08,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:10,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:12,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:14,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:16,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:16,775 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:14:18,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:20,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:14:20,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:14:22,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:24,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:26,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:28,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:30,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:32,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:34,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:36,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:14:36,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:14:36,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:38,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:41,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:43,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:45,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:46,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:14:47,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:49,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:51,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:51,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:14:51,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:14:53,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:55,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:57,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:14:59,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:01,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:03,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:05,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:06,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:15:06,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:15:07,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:09,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:11,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:13,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:15,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:16,912 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:15:17,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:19,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:21,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:15:22,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:15:23,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:25,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:27,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:29,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:31,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:33,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:35,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:37,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:37,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:15:37,258 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:15:39,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:41,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:43,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:45,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:46,988 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:15:47,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:49,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:51,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:15:52,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:15:53,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:55,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:57,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:15:59,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:01,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:04,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:06,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:07,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:16:07,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:16:08,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:10,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:12,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:14,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:16,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:17,066 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:16:18,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:20,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:22,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:23,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:16:23,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:16:24,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:26,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:28,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:30,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:32,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:34,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:36,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:38,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:38,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:16:38,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:16:40,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:42,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:44,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:46,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:47,145 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:16:48,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:50,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:52,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:16:54,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:16:54,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:56,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:16:58,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:00,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:02,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:04,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:06,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:08,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:09,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:17:09,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:17:10,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:12,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:14,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:16,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:17,210 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:17:18,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:20,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:22,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:24,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:17:24,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:17:26,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:28,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:30,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:32,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:34,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:36,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:38,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:39,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:17:39,856 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:17:40,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:42,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:44,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:46,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:47,289 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:17:48,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:50,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:52,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:55,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:17:55,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:17:55,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:57,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:17:59,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:01,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:03,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:05,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:07,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:09,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:18:10,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:18:11,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:13,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:15,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:17,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:17,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:18:19,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:21,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:23,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:25,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:18:25,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:18:27,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:29,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:31,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:33,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:35,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:37,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:39,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:18:41,012 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:18:41,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:43,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:45,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:47,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:47,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:18:49,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:51,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:53,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:55,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:56,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:18:56,245 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:18:57,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:18:59,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:01,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:03,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:05,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:07,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:09,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:11,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:19:11,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:19:13,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:15,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:17,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:17,521 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:19:19,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:21,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:23,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:25,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:26,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:19:26,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:19:28,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:30,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:32,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:34,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:36,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:38,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:40,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:42,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:19:42,029 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:19:42,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:44,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:46,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:47,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:19:48,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:50,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:52,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:54,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:56,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:19:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:19:57,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:19:58,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:00,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:02,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:04,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:06,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:08,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:10,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:12,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:12,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:20:12,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:20:14,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:16,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:17,669 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:20:18,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:20,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:22,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:24,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:26,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:27,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:20:27,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:20:28,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:30,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:32,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:34,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:36,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:38,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:40,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:42,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:43,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:20:43,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:20:44,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:46,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:47,742 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:20:48,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:50,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:52,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:54,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:56,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:58,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:20:58,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:20:58,557 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:21:00,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:02,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:04,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:06,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:08,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:10,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:12,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:13,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:21:13,797 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:21:14,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:16,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:17,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:21:18,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:20,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:22,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:24,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:26,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:28,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:29,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:21:29,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:21:30,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:32,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:34,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:36,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:38,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:40,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:43,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:44,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:21:44,375 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:21:45,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:47,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:47,883 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:21:49,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:51,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:53,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:55,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:57,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:59,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:21:59,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:21:59,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:22:01,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:03,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:05,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:07,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:09,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:11,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:13,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:14,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:22:14,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:22:15,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:17,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:17,952 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:22:19,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:21,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:23,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:25,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:27,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:29,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:22:30,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:22:31,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:33,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:35,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:37,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:39,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:41,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:43,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:45,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:45,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:22:45,473 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:22:47,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:48,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:22:49,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:51,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:53,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:55,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:57,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:22:59,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:00,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:23:00,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:23:01,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:03,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:05,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:07,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:09,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:11,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:14,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:16,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:23:16,040 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:23:16,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:18,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:23:18,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:20,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:22,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:24,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:26,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:28,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:30,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:31,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:23:31,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:23:32,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:34,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:36,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:38,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:40,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:42,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:44,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:46,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:46,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:23:46,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:23:48,167 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:23:48,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:50,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:52,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:54,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:56,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:23:58,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:00,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:01,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:24:01,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:24:02,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:04,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:06,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:08,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:10,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:12,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:14,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:16,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:17,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:24:17,065 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:24:18,239 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:24:18,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:20,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:22,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:24,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:26,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:28,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:30,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:24:32,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:24:32,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:34,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:36,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:38,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:40,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:42,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:44,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:46,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:47,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:24:47,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:24:48,304 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:24:48,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:50,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:52,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:55,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:57,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:24:59,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:01,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:03,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:25:03,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:25:03,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:05,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:07,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:09,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:11,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:13,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:15,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:17,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:18,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:25:18,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:25:18,455 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:25:19,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:21,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:23,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:25,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:27,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:29,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:31,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:33,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:33,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:25:33,497 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:25:35,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:37,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:39,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:41,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:43,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:45,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:47,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:48,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:25:48,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:25:48,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:25:49,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:51,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:53,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:55,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:57,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:25:59,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:01,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:03,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:04,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:26:04,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:26:05,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:07,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:09,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:11,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:13,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:15,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:17,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:18,515 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:26:19,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:26:19,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:26:20,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:22,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:24,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:26,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:28,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:30,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:32,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:34,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:26:34,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:26:36,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:38,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:40,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:42,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:44,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:46,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:48,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:48,582 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:26:50,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:26:50,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:26:50,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:52,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:54,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:56,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:26:58,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:00,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:02,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:04,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:05,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:27:05,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:27:06,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:08,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:10,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:12,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:14,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:16,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:18,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:18,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:27:20,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:20,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:27:20,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:27:22,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:24,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:26,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:28,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:30,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:32,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:34,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:35,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:27:35,829 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:27:36,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:39,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:41,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:43,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:45,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:47,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:48,725 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:27:49,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:51,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:27:51,056 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:27:51,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:53,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:27:55,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:28:06,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:28:06,366 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:28:18,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:28:21,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:28:21,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:28:36,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:28:36,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:28:48,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:28:52,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:28:52,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:28:56,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:28:56,300 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 19:28:56,354 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 19:28:56,430 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 19:28:56,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 19:28:57,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:29:03,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:29:07,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:29:07,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:29:18,991 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:29:19,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:29:22,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:29:22,653 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:29:37,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:29:37,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:29:49,062 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:29:53,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:29:53,099 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:30:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:30:08,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:30:19,137 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:30:23,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:30:23,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:30:38,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:30:38,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:30:49,211 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:30:54,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:30:54,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:31:09,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:31:09,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:31:19,286 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:31:24,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:31:24,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:31:39,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:31:39,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:31:49,361 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:31:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:31:54,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:32:10,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:32:10,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:32:19,439 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:32:25,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:32:25,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:32:40,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:32:40,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:32:49,519 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:32:55,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:32:55,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:33:09,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:11,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:33:11,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:33:11,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:11,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:13,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:13,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:14,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:16,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:17,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:19,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:19,595 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:33:19,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:21,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:21,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:23,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:25,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:26,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:33:26,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:33:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:27,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:28,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:29,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:31,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:31,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:33,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:33,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:35,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:35,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:37,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:39,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:40,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:40,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:41,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:33:41,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:33:42,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:44,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:44,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:44,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:46,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:48,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:49,670 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:33:50,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:52,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:54,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:56,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:33:56,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:33:56,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:33:56,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:33:58,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:00,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:10,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:12,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:34:12,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:34:14,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:18,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:19,745 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:34:22,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:26,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:27,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:34:27,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:34:30,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:33,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:33,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:34,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:36,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:40,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:34:42,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:34:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:43,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:44,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:46,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:49,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:49,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:34:50,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:52,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:52,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:54,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:55,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:55,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:56,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:57,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:34:57,866 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:34:58,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:34:58,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:34:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:00,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:02,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:03,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:03,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:03,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:06,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:08,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:08,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:10,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:11,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:11,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:12,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:35:13,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:35:13,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:14,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:18,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:19,886 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:35:20,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:20,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:22,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:23,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:24,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:26,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:28,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:35:28,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:35:28,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:29,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:29,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:30,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:31,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:31,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:32,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:34,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:35,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:36,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:37,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:37,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:38,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:39,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:39,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:40,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:41,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:42,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:43,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:35:43,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:35:44,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:46,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:48,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:49,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:35:50,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:53,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:54,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:54,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:35:55,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:57,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:35:58,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:35:58,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:36:05,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:11,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:14,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:36:14,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:36:15,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:19,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:20,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:36:23,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:27,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:29,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:36:29,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:36:29,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:32,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:33,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:35,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:39,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:41,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:44,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:36:44,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:36:45,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:45,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:47,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:50,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:36:51,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:53,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:55,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:56,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:56,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:57,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:58,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:36:59,425 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 19:36:59,641 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 19:36:59,704 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 19:36:59,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 19:36:59,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:36:59,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:36:59,828 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:37:00,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:00,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:01,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:03,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:04,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:04,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:05,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:07,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:09,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:09,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:10,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:11,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:12,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:13,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:15,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:37:15,137 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:37:15,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:17,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:19,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:20,180 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:37:20,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:20,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:21,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:22,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:25,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:27,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:28,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:28,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:29,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:29,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:30,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:37:30,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:37:31,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:31,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:33,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:34,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:34,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:35,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:36,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:36,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:37,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:38,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:38,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:39,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:39,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:40,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:37:41,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:43,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:45,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:37:45,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:37:45,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:47,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:49,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:37:50,254 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:37:59,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:00,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:38:00,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:38:02,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:02,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:03,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:07,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:11,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:15,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:16,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:38:16,206 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:38:19,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:20,317 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:38:22,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:26,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:28,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:31,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:38:31,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:38:32,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:34,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:38,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:40,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:42,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:44,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:46,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:38:46,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:38:46,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:47,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:48,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:50,385 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:38:50,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:50,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:52,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:53,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:54,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:55,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:55,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:38:56,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:38:58,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:00,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:01,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:01,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:39:02,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:39:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:02,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:03,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:04,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:05,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:05,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:06,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:07,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:07,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:08,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:10,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:11,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:12,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:14,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:14,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:16,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:17,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:39:17,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:39:17,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:17,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:18,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:18,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:20,462 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:39:20,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:22,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:22,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:24,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:26,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:27,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:28,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:30,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:30,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:32,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:39:32,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:39:32,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:32,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:32,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:33,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:34,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:36,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:38,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:40,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:42,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:39:47,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:39:50,530 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:39:50,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:39:54,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:39:58,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:02,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:40:03,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:40:06,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:10,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:14,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:16,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:18,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:40:18,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:40:20,600 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:40:20,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:22,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:27,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:29,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:40:33,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:40:33,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:35,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:37,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:38,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:39,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:41,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:43,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:43,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:43,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:44,063 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 19:40:44,263 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 19:40:44,330 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 19:40:44,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 19:40:45,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:47,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:48,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:48,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:40:48,794 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:40:50,752 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:40:50,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:51,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:52,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:52,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:53,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:54,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:54,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:55,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:56,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:57,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:40:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:40:59,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:00,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:01,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:02,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:02,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:03,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:04,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:41:04,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:41:04,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:04,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:05,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:07,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:07,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:08,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:09,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:11,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:11,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:11,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:12,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:13,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:15,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:15,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:17,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:17,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:19,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:41:19,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:41:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:19,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:20,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:20,825 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:41:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:21,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:23,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:25,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:26,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:27,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:29,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:29,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:31,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:32,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:41:33,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:34,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:41:34,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:41:35,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:37,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:47,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:49,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:41:49,886 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:41:50,893 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:41:53,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:41:57,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:01,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:05,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:42:05,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:42:05,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:05,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:09,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:13,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:15,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:19,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:20,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:42:20,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:42:20,961 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:42:21,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:25,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:27,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:31,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:33,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:35,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:42:35,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:42:35,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:39,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:41,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:42,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:42,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:43,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:44,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:44,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:45,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:47,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:49,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:49,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:42:50,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:42:51,208 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:42:51,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:53,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:54,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:56,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:57,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:57,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:42:58,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:58,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:42:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:00,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:01,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:01,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:02,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:04,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:05,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:05,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:43:06,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:43:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:06,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:07,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:07,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:08,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:09,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:09,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:10,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:11,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:12,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:13,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:14,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:14,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:16,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:17,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:18,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:19,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:20,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:21,103 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:43:21,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:43:21,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:43:22,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:24,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:26,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:27,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:28,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:29,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:29,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:30,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:31,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:31,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:43:32,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:34,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:36,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:43:36,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:43:42,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:46,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:51,179 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:43:51,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:43:51,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:43:52,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:54,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:43:58,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:02,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:02,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:06,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:44:07,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:44:10,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:12,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:16,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:18,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:21,258 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:44:22,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:44:22,542 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:44:22,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:24,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:26,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:30,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:32,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:34,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:35,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:36,259 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 19:44:36,465 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 19:44:36,534 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 19:44:36,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 19:44:37,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:44:37,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:44:37,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:38,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:40,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:42,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:43,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:44,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:46,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:47,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:47,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:48,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:50,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:51,371 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:44:52,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:52,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:53,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:44:53,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:44:54,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:54,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:55,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:56,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:57,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:57,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:58,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:44:59,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:44:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:00,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:02,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:03,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:03,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:04,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:05,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:06,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:08,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:08,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:45:08,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:45:08,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:09,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:10,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:12,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:13,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:13,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:14,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:16,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:18,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:21,442 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:45:21,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:22,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:22,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:45:23,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:45:23,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:24,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:24,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:25,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:27,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:35,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:45:37,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:38,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:45:38,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:45:43,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:47,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:51,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:45:51,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:53,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:45:53,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:45:55,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:45:59,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:03,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:05,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:46:09,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:46:09,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:11,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:15,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:17,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:21,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:46:21,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:23,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:24,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:46:24,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:46:25,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:29,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:31,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:31,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:31,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:32,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:32,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:33,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:35,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:37,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:37,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:39,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:46:39,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:46:39,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:40,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:40,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:44,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:44,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:46,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:46,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:48,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:50,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:51,675 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:46:52,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:52,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:54,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:46:54,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:46:55,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:55,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:56,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:56,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:57,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:46:58,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:46:59,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:00,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:00,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:01,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:02,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:02,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:03,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:04,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:04,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:06,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:06,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:08,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:08,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:10,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:47:10,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:47:10,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:11,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:12,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:14,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:15,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:16,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:17,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:17,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:18,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:18,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:20,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:21,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:21,746 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:47:22,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:22,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:22,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:24,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:25,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:47:25,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:47:33,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:39,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:40,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:47:40,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:47:42,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:47:43,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:47,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:51,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:51,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:47:53,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:47:56,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:47:56,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:47:57,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:01,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:03,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:03,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:07,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:09,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:11,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:48:11,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:48:13,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:15,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:17,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:21,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:21,897 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:48:23,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:25,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:26,065 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 19:48:26,293 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 19:48:26,370 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 19:48:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:48:26,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:48:27,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 19:48:27,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:31,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:33,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:34,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:34,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:35,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:35,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:37,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:39,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:41,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:41,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:41,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:48:41,830 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:48:43,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:43,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:43,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:45,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:47,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:49,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:50,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:51,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:52,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:48:53,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:53,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:54,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:55,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:55,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:57,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:48:57,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:48:57,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:58,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:58,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:48:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:48:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:00,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:01,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:01,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:02,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:03,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:04,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:05,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:06,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:07,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:09,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:11,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:12,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:49:12,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:49:13,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:14,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:15,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:15,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:17,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:19,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:22,108 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:49:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:27,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:49:27,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:49:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:29,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:31,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:36,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:40,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:42,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:49:42,827 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:49:44,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:45,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:45,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:49:48,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:52,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:52,186 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:49:54,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:49:58,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:49:58,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:49:58,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:00,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:04,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:04,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:04,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:08,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:10,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:12,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:50:13,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:50:16,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:18,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:19,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:19,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:19,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:19,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:20,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:22,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:22,263 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:50:24,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:26,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:28,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:50:28,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:50:30,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:31,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:32,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:34,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:36,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:36,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:38,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:40,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:42,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:50:43,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:50:43,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:44,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:45,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:46,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:46,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:48,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:48,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:49,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:50,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:51,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:52,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:52,341 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:50:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:56,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:50:58,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:50:59,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:50:59,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:50:59,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:00,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:00,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:01,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:02,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:02,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:03,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:04,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:06,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:06,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:08,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:10,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:12,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:51:14,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:51:17,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:17,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:20,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:22,415 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:51:24,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:26,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:29,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:51:29,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:51:30,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:32,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:33,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:33,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:36,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:40,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:44,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:45,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:51:45,035 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:51:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:45,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:48,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:50,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:52,492 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:51:54,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:56,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:51:59,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:51:59,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:00,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:52:00,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:52:02,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:06,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:08,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:11,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:13,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 19:52:14,504 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 19:52:14,717 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 19:52:14,779 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 19:52:15,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 19:52:15,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:52:15,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:52:16,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:17,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:19,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:21,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:22,801 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:52:23,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:25,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:27,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:29,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:30,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:52:30,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:52:31,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:33,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:35,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:37,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:39,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:41,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:43,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:45,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:46,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:52:46,079 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:52:47,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:49,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:51,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:52,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:52:53,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:55,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:57,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:52:59,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:01,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:53:01,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:53:03,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:05,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:07,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:09,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:11,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:13,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:15,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:16,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:53:16,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:53:17,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:19,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:21,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:22,958 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:53:24,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:26,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:28,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:30,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:53:32,122 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:53:32,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:34,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:36,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:38,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:40,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:42,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:44,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:46,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:47,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:53:47,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:53:48,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:50,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:52,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:53,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:53:54,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:56,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:53:58,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:00,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:02,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:54:02,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:54:04,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:06,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:08,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:10,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:12,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:14,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:16,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:18,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:54:18,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:54:18,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:20,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:22,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:23,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:54:24,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:26,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:28,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:30,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:32,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:33,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:54:33,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:54:34,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:36,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:38,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:40,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:42,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:44,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:47,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:48,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:54:48,742 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:54:49,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:51,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:53,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:54:53,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:55,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:57,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:54:59,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:01,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:03,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:03,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:55:03,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:55:05,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:07,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:09,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:11,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:13,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:15,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:17,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:19,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:55:19,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:55:19,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:21,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:23,253 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:55:23,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:25,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:27,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:29,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:31,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:33,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:34,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:55:34,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:55:35,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:37,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:39,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:41,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:43,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:45,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:47,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:49,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:55:49,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:55:51,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:53,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:55:53,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:55,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:57,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:55:59,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:01,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:03,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:05,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:56:05,057 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:56:05,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:07,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:09,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:11,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:13,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:15,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:17,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:19,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:20,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:56:20,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:56:21,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:23,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:23,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:56:25,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:27,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:29,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:31,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:34,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:35,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:56:35,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:56:36,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:38,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:40,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:42,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:44,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:46,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:48,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:50,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:56:50,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:56:52,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:53,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:56:54,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:56,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:56:58,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:00,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:02,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:04,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:57:06,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:57:06,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:08,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:10,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:12,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:14,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:16,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:18,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:20,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:21,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:57:21,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:57:22,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:23,544 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:57:24,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:26,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:28,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:30,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:32,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:34,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:36,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:36,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:57:36,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:57:38,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:40,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:42,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:44,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:46,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:48,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:50,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:52,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:57:52,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:57:52,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:53,612 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:57:55,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:57,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:57:59,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:01,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:03,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:05,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:58:07,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:58:07,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:09,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:11,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:13,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:15,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:17,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:19,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:21,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:58:22,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:58:23,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:23,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:58:25,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:27,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:29,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:31,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:33,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:35,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:37,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:58:37,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:58:39,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:41,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:43,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:45,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:47,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:49,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:51,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:53,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:58:53,091 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:58:53,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:53,764 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:58:55,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:57,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:58:59,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:01,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:03,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:05,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:07,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:08,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:59:08,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:59:09,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:11,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:13,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:15,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:17,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:19,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:21,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:23,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:23,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:59:23,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:59:23,835 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:59:25,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:27,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:29,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:31,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:33,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:35,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:38,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:38,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:59:38,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:59:40,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:42,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:44,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:46,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:48,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:50,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:52,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:53,903 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 19:59:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 19:59:54,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 19:59:54,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:56,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 19:59:58,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:00,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:02,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:04,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:06,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:08,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:00:09,336 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:00:10,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:12,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:14,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:16,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:18,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:20,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:22,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:23,978 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:00:24,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:24,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:00:24,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:00:26,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:28,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:30,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:32,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:34,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:36,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:38,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:39,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:00:39,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:00:40,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:42,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:44,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:46,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:48,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:50,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:52,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:54,041 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:00:54,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:55,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:00:55,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:00:56,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:00:58,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:00,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:02,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:05,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:07,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:09,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:10,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:01:10,639 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:01:11,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:13,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:15,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:17,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:19,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:21,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:23,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:24,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:01:25,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:25,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:01:25,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:01:27,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:29,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:31,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:33,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:35,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:37,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:39,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:01:41,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:01:41,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:43,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:45,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:47,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:49,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:51,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:53,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:54,192 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:01:55,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:56,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:01:56,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:01:57,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:01:59,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:01,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:03,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:05,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:07,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:09,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:11,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:11,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:02:11,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:02:13,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:15,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:17,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:19,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:22,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:24,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:02:24,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:26,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:26,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:02:26,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:02:28,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:30,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:32,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:34,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:36,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:38,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:40,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:02:42,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:02:42,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:44,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:46,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:48,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:50,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:52,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:54,338 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:02:54,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:56,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:02:57,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:02:57,540 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:02:58,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:00,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:02,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:04,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:06,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:08,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:10,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:12,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:12,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:03:12,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:03:14,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:16,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:18,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:20,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:22,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:24,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:03:24,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:26,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:28,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:03:28,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:03:28,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:30,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:32,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:34,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:36,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:38,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:40,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:42,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:43,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:03:43,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:03:44,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:46,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:48,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:50,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:53,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:54,479 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:03:55,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:57,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:03:58,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:03:58,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:03:59,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:01,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:03,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:05,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:07,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:09,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:11,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:13,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:04:14,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:04:15,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:17,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:19,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:21,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:23,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:24,574 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:04:25,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:27,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:29,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:04:29,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:04:29,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:31,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:33,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:35,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:37,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:39,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:41,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:43,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:04:44,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:04:45,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:47,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:49,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:51,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:53,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:54,627 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:04:55,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:57,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:04:59,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:00,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:05:00,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:05:01,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:03,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:05,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:07,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:09,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:11,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:13,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:05:15,409 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:05:15,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:17,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:19,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:21,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:23,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:24,697 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:05:25,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:27,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:29,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:30,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:05:30,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:05:31,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:33,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:35,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:37,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:39,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:41,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:43,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:45,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:46,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:05:46,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:05:47,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:49,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:51,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:53,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:54,770 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:05:56,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:05:58,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:00,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:01,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:06:01,293 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:06:02,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:04,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:06,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:08,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:10,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:12,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:14,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:16,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:06:16,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:06:16,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:18,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:20,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:22,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:24,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:24,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:06:26,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:28,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:30,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:06:31,819 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:06:32,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:34,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:36,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:38,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:40,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:42,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:44,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:46,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:47,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:06:47,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:06:48,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:50,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:52,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:54,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:54,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:06:56,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:06:58,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:00,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:02,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:07:02,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:07:02,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:04,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:06,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:08,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:10,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:13,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:15,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:17,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:17,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:07:17,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:07:19,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:21,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:23,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:24,965 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:07:25,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:27,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:29,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:31,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:32,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:07:32,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:07:33,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:35,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:37,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:39,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:41,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:43,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:45,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:47,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:48,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:07:48,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:07:49,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:51,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:53,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:55,040 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:07:55,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:57,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:07:59,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:01,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:08:03,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:08:03,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:05,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:07,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:09,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:11,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:13,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:15,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:17,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:18,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:08:18,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:08:19,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:21,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:23,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:25,109 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:08:25,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:27,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:29,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:31,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:33,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:34,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:08:34,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:08:35,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:37,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:39,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:41,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:43,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:45,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:47,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:49,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:08:49,321 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:08:49,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:51,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:53,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:55,180 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:08:55,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:57,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:08:59,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:01,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:03,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:04,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:09:04,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:09:05,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:07,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:09,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:11,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:13,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:15,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:18,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:09:19,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:09:20,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:22,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:24,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:25,251 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:09:26,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:28,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:30,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:32,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:34,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:09:35,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:09:36,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:38,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:40,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:42,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:44,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:46,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:48,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:09:50,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:09:50,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:52,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:54,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:55,315 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:09:56,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:09:58,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:00,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:02,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:04,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:05,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:10:05,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:10:06,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:08,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:10,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:12,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:14,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:16,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:18,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:20,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:10:21,428 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:10:22,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:24,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:25,386 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:10:26,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:28,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:30,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:32,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:34,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:36,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:10:36,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:10:36,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:38,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:40,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:42,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:44,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:46,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:48,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:50,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:51,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:10:51,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:10:52,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:54,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:55,462 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:10:56,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:10:58,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:00,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:02,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:04,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:06,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:11:07,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:11:08,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:10,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:13,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:15,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:17,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:19,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:21,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:22,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:11:22,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:11:23,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:25,535 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:11:25,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:27,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:29,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:31,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:33,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:35,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:37,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:11:37,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:11:37,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:39,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:41,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:43,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:45,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:47,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:49,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:51,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:52,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:11:52,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:11:53,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:55,610 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:11:55,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:57,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:11:59,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:01,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:03,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:05,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:07,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:08,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:12:08,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:12:09,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:11,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:13,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:15,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:17,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:19,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:21,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:23,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:12:23,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:12:23,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:25,684 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:12:25,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:27,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:29,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:31,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:33,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:35,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:37,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:12:38,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:12:39,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:41,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:44,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:46,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:48,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:50,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:52,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:54,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:12:54,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:12:54,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:55,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:12:56,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:12:58,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:00,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:02,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:04,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:06,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:08,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:13:09,461 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:13:10,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:12,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:14,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:16,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:18,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:20,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:22,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:24,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:13:24,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:13:24,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:25,832 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:13:26,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:28,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:30,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:32,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:34,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:36,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:38,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:39,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:13:39,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:13:40,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:42,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:44,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:46,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:48,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:50,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:52,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:54,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:55,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:13:55,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:13:55,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:13:56,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:13:58,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:00,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:02,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:04,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:06,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:08,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:10,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:14:10,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:14:10,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:12,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:14,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:16,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:18,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:20,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:22,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:24,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:25,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:14:25,757 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:14:25,998 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:14:26,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:28,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:30,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:32,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:34,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:36,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:38,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:40,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:41,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:14:41,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:14:42,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:44,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:46,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:48,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:51,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:53,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:55,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:56,074 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:14:56,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:14:56,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:14:57,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:14:59,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:01,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:03,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:05,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:07,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:09,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:11,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:15:11,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:15:11,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:13,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:15,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:17,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:19,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:21,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:23,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:25,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:26,154 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:15:26,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:15:26,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:15:27,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:29,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:31,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:33,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:35,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:37,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:39,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:41,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:42,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:15:42,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:15:43,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:45,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:47,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:49,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:51,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:53,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:55,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:56,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:15:57,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:15:57,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:15:57,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:15:59,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:01,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:03,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:05,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:07,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:09,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:11,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:12,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:16:12,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:16:13,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:15,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:17,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:19,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:21,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:23,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:25,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:26,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:16:27,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:28,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:16:28,066 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:16:29,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:31,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:33,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:35,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:38,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:40,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:42,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:43,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:16:43,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:16:44,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:46,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:48,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:50,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:52,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:54,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:56,384 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:16:56,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:16:58,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:16:58,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:16:58,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:00,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:02,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:04,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:06,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:08,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:10,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:12,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:13,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:17:13,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:17:14,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:16,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:18,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:20,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:22,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:24,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:26,461 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:17:26,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:28,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:29,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:17:29,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:17:30,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:32,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:34,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:36,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:38,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:40,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:42,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:44,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:17:44,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:17:44,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:46,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:48,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:50,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:52,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:54,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:56,536 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:17:56,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:58,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:17:59,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:17:59,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:18:00,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:02,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:05,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:07,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:09,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:11,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:13,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:14,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:18:14,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:18:15,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:17,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:19,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:21,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:23,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:25,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:26,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:18:27,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:29,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:30,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:18:30,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:18:31,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:33,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:35,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:37,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:39,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:41,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:43,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:45,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:18:45,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:18:45,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:47,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:49,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:51,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:53,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:55,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:56,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:18:57,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:18:59,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:00,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:19:00,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:19:01,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:04,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:06,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:08,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:10,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:12,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:14,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:16,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:19:16,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:19:16,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:18,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:20,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:22,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:24,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:26,767 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:19:26,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:28,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:30,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:31,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:19:31,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:19:32,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:34,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:36,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:38,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:40,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:42,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:44,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:46,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:19:46,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:19:46,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:48,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:50,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:52,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:54,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:56,843 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:19:56,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:19:58,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:00,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:01,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:20:01,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:20:02,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:04,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:06,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:08,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:10,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:12,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:14,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:16,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:20:16,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:20:16,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:19,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:21,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:23,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:25,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:26,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:20:27,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:29,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:31,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:32,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:20:32,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:20:33,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:35,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:37,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:39,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:41,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:43,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:45,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:47,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:20:47,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:20:47,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:49,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:51,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:53,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:55,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:20:56,997 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:20:58,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:00,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:02,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:02,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:21:02,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:21:04,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:06,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:08,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:10,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:12,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:14,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:16,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:18,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:18,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:21:18,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:21:20,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:22,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:24,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:26,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:27,074 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:21:28,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:30,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:32,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:33,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:21:33,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:21:34,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:36,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:38,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:40,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:42,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:44,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:47,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:48,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:21:48,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:21:49,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:51,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:53,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:55,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:57,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:21:57,154 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:21:59,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:01,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:03,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:03,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:22:03,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:22:05,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:07,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:09,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:11,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:13,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:15,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:17,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:19,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:19,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:22:19,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:22:21,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:23,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:25,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:27,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:27,234 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:22:29,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:31,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:33,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:34,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:22:34,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:22:35,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:37,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:39,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:41,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:43,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:45,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:47,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:49,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:22:49,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:22:51,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:53,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:55,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:57,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:22:57,310 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:22:59,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:02,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:04,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:23:05,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:23:06,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:08,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:10,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:12,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:14,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:16,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:18,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:20,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:20,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:23:20,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:23:22,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:24,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:26,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:27,384 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:23:28,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:30,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:32,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:34,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:35,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:23:35,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:23:36,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:38,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:40,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:42,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:44,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:46,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:48,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:50,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:23:50,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:23:52,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:54,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:56,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:23:57,460 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:23:58,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:00,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:02,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:04,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:06,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:06,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:24:06,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:24:08,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:10,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:12,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:14,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:16,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:18,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:20,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:24:21,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:24:22,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:24,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:26,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:27,527 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:24:28,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:30,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:32,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:35,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:36,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:24:36,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:24:37,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:39,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:41,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:43,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:45,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:47,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:49,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:51,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:24:52,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:24:53,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:55,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:57,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:24:57,611 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:24:59,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:01,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:03,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:05,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:07,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:25:07,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:25:09,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:11,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:13,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:15,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:17,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:19,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:21,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:22,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:25:22,664 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:25:23,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:25,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:27,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:27,689 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:25:29,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:31,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:33,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:35,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:37,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:25:38,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:25:39,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:41,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:43,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:45,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:47,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:49,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:51,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:53,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:53,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:25:53,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:25:55,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:57,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:25:57,764 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:25:59,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:01,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:03,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:05,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:07,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:08,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:26:08,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:26:09,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:11,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:13,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:15,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:17,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:19,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:21,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:23,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:23,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:26:23,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:26:25,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:27,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:27,839 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:26:29,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:31,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:33,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:35,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:37,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:39,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:26:39,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:26:39,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:42,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:44,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:46,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:48,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:50,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:52,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:54,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:26:54,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:26:56,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:26:57,915 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:26:58,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:00,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:02,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:04,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:06,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:08,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:09,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:27:09,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:27:10,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:12,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:14,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:16,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:18,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:20,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:22,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:24,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:24,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:27:24,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:27:26,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:27,993 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:27:28,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:30,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:32,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:34,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:36,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:38,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:27:40,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:27:40,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:42,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:44,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:46,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:48,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:50,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:52,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:54,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:27:55,457 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:27:56,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:27:58,070 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:27:58,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:00,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:02,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:04,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:06,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:08,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:10,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:28:10,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:28:11,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:13,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:15,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:17,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:19,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:21,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:23,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:25,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:28:26,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:28:27,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:28,141 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:28:29,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:31,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:33,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:35,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:37,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:39,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:41,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:41,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:28:41,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:28:43,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:45,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:47,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:49,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:51,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:53,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:55,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:28:56,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:28:57,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:28:58,205 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:28:59,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:01,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:03,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:05,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:07,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:09,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:11,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:12,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:29:12,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:29:13,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:16,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:18,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:20,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:22,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:24,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:26,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:29:27,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:29:28,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:28,274 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:29:30,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:32,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:34,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:36,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:38,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:40,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:42,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:42,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:29:42,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:29:44,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:46,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:48,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:50,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:52,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:54,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:56,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:57,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:29:57,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:29:58,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:29:58,345 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:30:00,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:02,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:04,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:06,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:08,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:10,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:12,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:13,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:30:13,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:30:14,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:16,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:18,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:20,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:22,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:24,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:26,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:28,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:28,417 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:30:28,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:30:28,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:30:30,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:32,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:34,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:37,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:39,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:41,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:43,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:30:43,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:30:45,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:47,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:49,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:51,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:53,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:55,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:57,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:30:58,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:30:59,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:30:59,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:30:59,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:01,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:03,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:05,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:07,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:09,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:11,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:13,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:31:14,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:31:15,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:17,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:19,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:21,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:23,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:25,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:27,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:28,558 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:31:29,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:29,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:31:29,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:31:31,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:33,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:35,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:37,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:39,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:41,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:43,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:31:44,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:31:45,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:47,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:49,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:51,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:53,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:55,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:57,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:31:58,627 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:31:59,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:32:00,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:32:01,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:03,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:05,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:07,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:10,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:12,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:14,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:15,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:32:15,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:32:16,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:18,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:20,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:22,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:24,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:26,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:28,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:28,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:32:30,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:30,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:32:30,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:32:32,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:34,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:36,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:38,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:40,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:42,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:44,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:32:45,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:32:46,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:48,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:50,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:52,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:54,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:56,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:58,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:32:58,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:33:00,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:01,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:33:01,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:33:02,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:04,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:06,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:08,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:10,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:12,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:14,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:16,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:16,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:33:16,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:33:18,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:20,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:22,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:24,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:26,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:28,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:28,829 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:33:30,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:33:31,713 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:33:33,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:35,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:37,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:39,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:41,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:43,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:45,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:46,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:33:46,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:33:47,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:49,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:51,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:53,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:55,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:57,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:33:58,896 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:33:59,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:34:01,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:34:02,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:34:02,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:34:03,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:34:05,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:34:07,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:34:09,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:34:17,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:34:17,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:34:28,970 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:34:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:34:32,959 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:34:48,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:34:48,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:34:59,045 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:35:03,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:35:03,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:35:09,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:35:09,060 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 20:35:09,113 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 20:35:09,187 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 20:35:09,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 20:35:11,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:35:15,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:35:18,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:35:18,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:35:29,240 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:35:31,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:35:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:35:34,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:35:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:35:49,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:35:59,319 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:36:04,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:36:04,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:36:19,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:36:19,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:36:29,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:36:35,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:36:35,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:36:50,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:36:50,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:36:59,464 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:37:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:37:05,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:37:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:37:20,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:37:29,538 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:37:35,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:37:35,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:37:51,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:37:51,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:37:59,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:38:06,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:38:06,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:38:21,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:38:21,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:38:29,692 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:38:36,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:38:36,845 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:38:52,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:38:52,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:38:59,769 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:39:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:39:07,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:39:22,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:39:22,599 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:39:24,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:28,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:29,847 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:39:30,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:32,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:34,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:36,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:37,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:39:37,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:39:38,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:38,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:40,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:42,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:44,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:46,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:47,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:47,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:48,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:50,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:50,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:52,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:53,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:39:53,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:39:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:53,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:54,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:56,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:57,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:57,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:58,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:39:59,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:39:59,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:40:00,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:00,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:01,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:02,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:03,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:04,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:04,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:06,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:08,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:40:08,421 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:40:08,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:08,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:08,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:10,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:11,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:12,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:12,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:14,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:20,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:23,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:40:23,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:40:24,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:29,999 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:40:30,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:34,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:38,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:39,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:40:39,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:40:41,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:42,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:45,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:46,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:48,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:52,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:54,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:40:54,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:40:55,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:55,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:57,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:59,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:40:59,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:40:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:00,068 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:41:03,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:05,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:07,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:09,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:41:09,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:41:10,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:10,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:11,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:13,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:15,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:17,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:18,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:21,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:23,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:41:25,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:41:25,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:27,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:29,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:30,143 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:41:31,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:33,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:35,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:37,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:37,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:37,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:39,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:40,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:41:40,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:41:40,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:41,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:43,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:45,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:47,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:47,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:49,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:49,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:50,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:50,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:51,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:53,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:53,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:54,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:55,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:41:55,596 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:41:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:56,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:41:57,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:57,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:58,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:41:59,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:00,218 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:42:00,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:00,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:02,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:03,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:05,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:06,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:08,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:09,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:10,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:42:10,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:42:11,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:13,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:17,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:19,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:24,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:25,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:26,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:42:26,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:42:29,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:30,296 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:42:31,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:35,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:36,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:39,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:40,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:40,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:42:41,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:42:42,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:42,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:43,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:44,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:42:45,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:49,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:51,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:55,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:42:56,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:42:56,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:42:58,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:00,365 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:43:01,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:01,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:02,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:03,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:03,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:04,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:06,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:08,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:10,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:11,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:11,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:11,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:11,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:11,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:43:11,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:43:12,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:12,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:12,804 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 20:43:13,003 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 20:43:13,076 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 20:43:13,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 20:43:14,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:16,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:18,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:22,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:24,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:26,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:27,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:43:27,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:43:28,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:30,459 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:43:30,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:32,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:33,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:34,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:36,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:38,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:40,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:42,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:43:42,479 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:43:42,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:43,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:44,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:46,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:47,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:47,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:48,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:49,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:49,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:50,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:51,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:51,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:52,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:53,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:54,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:55,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:56,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:56,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:43:57,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:43:58,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:43:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:43:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:00,533 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:44:00,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:01,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:01,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:02,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:02,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:04,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:12,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:44:12,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:44:14,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:14,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:18,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:18,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:20,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:21,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:22,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:23,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:26,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:28,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:44:28,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:44:30,609 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:44:30,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:31,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:34,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:36,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:36,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:38,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:41,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:42,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:43,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:44:43,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:44:44,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:45,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:44:48,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:50,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:54,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:56,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:44:58,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:44:58,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:45:00,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:00,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:45:02,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:02,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:04,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:08,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:10,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:12,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:14,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:45:14,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:45:14,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:15,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:15,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:18,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:20,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:22,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:24,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:26,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:28,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:45:29,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:45:30,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:30,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:45:32,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:34,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:36,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:38,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:39,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:40,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:42,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:43,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:43,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:44,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:44,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:45:44,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:45:46,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:46,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:46,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:48,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:50,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:50,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:51,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:53,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:54,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:55,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:56,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:57,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:58,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:58,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:45:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:45:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:45:59,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:46:00,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:00,835 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:46:01,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:03,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:09,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:09,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:12,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:15,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:46:15,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:46:15,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:18,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:18,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:19,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:20,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:20,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:22,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:23,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:27,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:28,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:29,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:30,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:46:30,599 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:46:30,912 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:46:31,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:31,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:33,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:37,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:39,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:40,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:40,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:43,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:45,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:45,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:45,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:46:45,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:46:46,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:46,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:46:49,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:51,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:53,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:57,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:46:59,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:00,988 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:47:01,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:47:01,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:47:01,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:02,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:02,872 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 20:47:03,076 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 20:47:03,136 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 20:47:03,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 20:47:03,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:05,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:07,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:08,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:08,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:09,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:11,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:11,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:13,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:15,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:16,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:47:16,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:47:17,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:19,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:21,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:23,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:25,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:27,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:29,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:31,208 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:47:31,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:47:31,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:47:31,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:32,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:35,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:36,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:36,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:37,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:39,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:39,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:39,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:41,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:42,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:43,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:45,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:46,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:47:46,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:47:46,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:46,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:47,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:48,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:48,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:49,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:50,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:50,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:51,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:52,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:52,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:47:53,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:47:55,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:01,291 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:48:02,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:48:02,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:48:03,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:05,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:05,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:07,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:08,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:11,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:12,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:12,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:13,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:14,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:15,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:16,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:48:17,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:48:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:17,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:18,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:19,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:21,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:23,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:27,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:27,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:29,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:31,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:48:32,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:48:32,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:48:33,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:35,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:36,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:36,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:39,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:41,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:41,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:43,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:45,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:47,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:47,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:48:47,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:48:49,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:54,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:55,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:55,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:55,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:55,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:48:56,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:48:58,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:00,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:00,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:01,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:49:02,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:49:03,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:49:04,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:07,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:07,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:08,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:10,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:12,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:14,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:16,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:18,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:49:18,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:49:18,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:20,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:22,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:24,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:26,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:28,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:30,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:31,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:31,517 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:49:32,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:32,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:33,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:33,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:49:33,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:49:34,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:35,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:36,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:38,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:39,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:39,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:40,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:41,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:42,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:43,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:44,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:45,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:46,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:48,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:49,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:49:49,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:49:49,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:49,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:56,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:49:57,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:49:59,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:01,593 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:50:02,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:50:04,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:50:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:06,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:08,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:09,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:10,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:12,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:12,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:14,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:15,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:17,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:18,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:19,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:50:19,528 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:50:20,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:20,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:21,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:24,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:24,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:28,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:30,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:31,663 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:50:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:50:34,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:50:34,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:36,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:36,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:37,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:40,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:42,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:44,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:44,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:46,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:47,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:50,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:50:50,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:50:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:50,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:50,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:50,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:50,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:51,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:50:51,316 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 20:50:51,522 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 20:50:51,594 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 20:50:51,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 20:50:52,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:54,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:56,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:50:58,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:00,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:01,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:51:03,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:04,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:05,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:51:05,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:51:06,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:08,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:08,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:09,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:10,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:13,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:15,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:17,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:19,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:51:20,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:51:21,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:23,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:25,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:27,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:29,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:29,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:30,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:30,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:31,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:31,834 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:51:32,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:32,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:33,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:35,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:51:35,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:51:35,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:37,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:38,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:38,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:39,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:40,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:40,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:41,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:43,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:44,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:46,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:51,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:51:51,037 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:51:52,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:52,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:53,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:56,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:56,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:57,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:51:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:51:58,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:00,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:01,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:52:03,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:52:06,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:52:06,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:06,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:07,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:11,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:12,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:12,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:13,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:16,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:16,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:17,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:18,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:20,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:52:21,497 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:52:21,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:21,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:23,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:27,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:29,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:29,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:31,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:31,985 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:52:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:35,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:36,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:52:36,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:52:37,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:37,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:38,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:38,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:39,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:41,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:41,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:42,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:43,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:45,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:45,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:45,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:46,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:46,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:52:47,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:49,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:51,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:51,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:52:51,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:52:53,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:57,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:52:59,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:01,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:02,062 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:53:03,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:05,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:53:07,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:53:07,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:09,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:11,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:13,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:15,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:17,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:19,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:21,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:53:22,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:53:23,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:24,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:24,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:26,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:28,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:30,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:30,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:31,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,138 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:53:32,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:32,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:33,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:33,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:34,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:35,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:36,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:37,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:53:37,887 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:53:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:38,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:44,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:45,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:46,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:47,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:50,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:52,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:53,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:53:53,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:53:54,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:54,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:56,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:53:58,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:53:59,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:00,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:01,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:01,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:02,216 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:54:04,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:04,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:06,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:06,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:08,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:54:08,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:54:08,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:10,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:12,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:12,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:14,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:16,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:17,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:18,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:19,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:19,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:20,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:23,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:54:23,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:54:24,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:26,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:27,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:28,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:30,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:31,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:31,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:31,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:32,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:54:32,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:34,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:35,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:37,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:38,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:54:39,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:54:39,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:40,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:40,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:40,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:40,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:40,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:41,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:41,294 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 20:54:41,492 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 20:54:41,561 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 20:54:41,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 20:54:42,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:44,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:46,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:47,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:54:48,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:52,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:54:54,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:54:54,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:56,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:54:58,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:00,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:02,420 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:55:02,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:04,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:06,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:06,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:07,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:08,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:09,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:55:09,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:55:10,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:13,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:15,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:17,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:19,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:21,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:21,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:21,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:22,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:23,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:24,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:55:24,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:55:25,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:25,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:27,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:29,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:29,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:30,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:31,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:32,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:55:33,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:55:40,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:55:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:41,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:43,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:48,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:49,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:51,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:53,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:55,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:55:55,324 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:55:56,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:57,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:55:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:55:58,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:00,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:01,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:01,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:01,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:02,576 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:56:03,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:06,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:07,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:10,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:56:10,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:56:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:10,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:11,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:13,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:16,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:17,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:18,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:18,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:19,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:21,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:22,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:23,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:56:25,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:56:25,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:27,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:27,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:29,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:31,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:31,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:32,653 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:56:33,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:35,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:36,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:36,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:37,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:39,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:40,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:41,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:56:41,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:56:41,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:43,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:44,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:45,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:46,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:46,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:47,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:49,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:56:51,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:53,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:55,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:56,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:56:56,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:56:57,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:56:59,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:01,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:02,732 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:57:04,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:05,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:06,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:07,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:08,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:10,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:11,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:57:11,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:57:12,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:14,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:16,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:18,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:20,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:22,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:24,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:26,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:27,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:57:27,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:57:27,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:27,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:28,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:32,809 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:57:36,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:42,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:57:42,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:57:42,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:44,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:45,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:47,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:48,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:49,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:52,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:53,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:53,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:56,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:57:57,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:57:57,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:57:58,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:57:58,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:00,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:01,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:01,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:58:02,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:02,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:02,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:03,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:06,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:08,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:12,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:13,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:58:13,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:58:15,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:15,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:17,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:18,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:19,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:21,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:23,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:24,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:25,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:27,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:28,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:58:28,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:58:28,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 20:58:29,872 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 20:58:30,086 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 20:58:30,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:30,142 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 20:58:31,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 20:58:31,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:32,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:32,968 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:58:34,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:36,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:38,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:40,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:42,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:43,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:58:43,542 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:58:44,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:46,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:48,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:50,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:52,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:54,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:56,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:58,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:58:58,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:58:58,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:59:00,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:02,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:03,044 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:59:04,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:06,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:08,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:10,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:12,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:14,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:14,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:59:14,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:59:16,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:18,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:20,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:22,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:24,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:26,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:28,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:29,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:59:29,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:59:30,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:32,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:33,119 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 20:59:34,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:36,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:38,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:40,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:42,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:44,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 20:59:44,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 20:59:46,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:48,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:51,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:53,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:55,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:57,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 20:59:59,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:00,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:00:00,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:00:01,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:03,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:03,189 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:00:05,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:07,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:09,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:11,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:13,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:15,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:15,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:00:15,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:00:17,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:19,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:21,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:23,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:25,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:27,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:29,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:30,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:00:30,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:00:31,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:33,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:33,270 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:00:35,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:37,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:39,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:41,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:43,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:45,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:45,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:00:45,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:00:47,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:49,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:51,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:53,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:55,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:57,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:00:59,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:01:01,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:01:01,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:03,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:03,348 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:01:05,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:07,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:09,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:11,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:13,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:16,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:16,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:01:16,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:01:18,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:20,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:22,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:24,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:26,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:28,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:30,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:31,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:01:31,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:01:32,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:33,425 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:01:34,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:36,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:38,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:40,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:42,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:44,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:46,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:01:46,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:01:48,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:50,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:52,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:54,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:56,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:01:58,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:00,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:02,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:02:02,163 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:02:02,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:03,490 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:02:04,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:06,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:08,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:10,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:12,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:14,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:16,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:17,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:02:17,388 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:02:18,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:20,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:22,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:24,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:26,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:28,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:30,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:32,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:32,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:02:32,687 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:02:33,557 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:02:34,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:36,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:39,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:41,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:43,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:45,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:47,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:47,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:02:47,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:02:49,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:51,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:53,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:55,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:57,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:02:59,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:01,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:03,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:03:03,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:03:03,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:03,633 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:03:05,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:07,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:09,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:11,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:13,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:15,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:17,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:18,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:03:18,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:03:19,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:21,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:23,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:25,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:27,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:29,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:31,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:33,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:33,712 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:03:33,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:03:33,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:03:35,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:37,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:39,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:41,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:43,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:45,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:47,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:49,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:03:49,083 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:03:49,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:51,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:53,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:55,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:57,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:03:59,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:01,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:03,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:03,788 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:04:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:04:04,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:04:05,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:07,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:09,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:11,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:13,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:15,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:17,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:19,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:04:19,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:04:21,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:23,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:25,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:27,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:29,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:31,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:33,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:33,864 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:04:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:04:35,065 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:04:35,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:37,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:39,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:41,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:43,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:45,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:47,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:49,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:04:50,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:04:52,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:54,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:56,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:04:58,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:00,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:02,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:03,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:05:04,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:05,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:05:05,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:05:06,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:08,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:10,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:12,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:14,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:16,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:18,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:20,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:20,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:05:20,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:05:22,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:24,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:26,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:28,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:30,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:32,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:34,018 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:05:34,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:36,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:05:36,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:05:36,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:38,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:40,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:42,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:44,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:46,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:48,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:50,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:51,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:05:51,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:05:52,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:54,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:56,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:05:58,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:00,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:02,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:04,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:06:04,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:06,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:06,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:06:06,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:06:08,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:10,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:12,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:14,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:16,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:18,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:20,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:21,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:06:21,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:06:22,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:24,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:26,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:28,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:30,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:32,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:34,170 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:06:34,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:36,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:06:37,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:06:38,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:40,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:42,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:44,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:46,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:48,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:51,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:06:52,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:06:53,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:55,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:57,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:06:59,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:01,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:03,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:04,245 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:07:05,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:07,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:07,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:07:07,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:07:09,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:11,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:13,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:15,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:17,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:19,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:21,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:22,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:07:22,971 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:07:23,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:25,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:27,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:29,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:31,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:33,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:34,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:07:35,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:37,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:38,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:07:38,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:07:39,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:41,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:43,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:45,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:47,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:49,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:51,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:53,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:53,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:07:53,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:07:55,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:57,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:07:59,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:01,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:03,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:04,397 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:08:05,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:07,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:08,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:08:08,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:08:09,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:11,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:13,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:15,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:17,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:19,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:22,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:24,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:08:24,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:08:24,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:26,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:28,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:30,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:32,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:34,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:34,476 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:08:36,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:38,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:39,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:08:39,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:08:40,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:42,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:44,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:46,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:48,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:50,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:52,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:54,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:08:54,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:08:56,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:08:58,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:00,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:02,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:04,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:04,553 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:09:06,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:08,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:10,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:09:10,040 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:09:10,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:12,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:14,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:16,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:18,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:20,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:22,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:24,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:25,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:09:25,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:09:26,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:28,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:30,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:32,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:34,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:34,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:09:36,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:38,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:40,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:09:40,575 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:09:41,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:43,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:45,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:47,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:49,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:51,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:53,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:55,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:55,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:09:55,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:09:57,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:09:59,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:01,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:03,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:04,708 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:10:05,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:07,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:09,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:11,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:10:11,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:10:11,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:13,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:15,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:17,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:19,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:21,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:23,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:25,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:10:26,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:10:27,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:29,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:31,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:33,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:34,782 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:10:35,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:37,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:39,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:41,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:41,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:10:41,572 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:10:43,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:45,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:48,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:50,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:52,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:54,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:56,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:10:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:10:56,886 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:10:58,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:00,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:02,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:04,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:04,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:11:06,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:08,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:10,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:11:12,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:11:12,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:14,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:16,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:18,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:20,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:22,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:24,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:26,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:11:27,461 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:11:28,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:30,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:32,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:34,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:34,930 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:11:36,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:38,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:40,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:42,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:42,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:11:42,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:11:44,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:46,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:48,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:50,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:52,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:55,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:57,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:11:57,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:11:58,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:11:59,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:01,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:03,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:05,007 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:12:05,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:07,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:09,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:11,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:13,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:12:13,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:12:13,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:15,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:17,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:19,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:21,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:23,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:25,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:27,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:28,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:12:28,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:12:29,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:31,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:33,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:35,086 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:12:35,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:37,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:39,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:41,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:43,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:43,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:12:43,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:12:45,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:47,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:49,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:51,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:53,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:55,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:57,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:12:59,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:12:59,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:12:59,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:01,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:03,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:05,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:13:05,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:07,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:09,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:11,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:13,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:14,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:13:14,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:13:15,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:17,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:19,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:21,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:23,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:25,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:27,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:29,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:29,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:13:29,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:13:32,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:34,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:35,238 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:13:36,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:38,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:40,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:42,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:44,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:45,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:13:45,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:13:46,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:48,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:50,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:52,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:54,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:56,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:13:58,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:14:00,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:14:00,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:02,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:04,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:05,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:14:06,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:08,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:10,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:12,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:14,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:15,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:14:15,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:14:16,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:18,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:20,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:22,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:24,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:26,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:28,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:30,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:31,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:14:31,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:14:32,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:34,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:35,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:14:36,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:38,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:40,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:42,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:44,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:46,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:14:46,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:14:46,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:48,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:50,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:52,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:54,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:56,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:14:58,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:00,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:01,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:15:01,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:15:02,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:04,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:05,466 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:15:07,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:09,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:11,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:13,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:15,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:15:16,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:15:17,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:19,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:21,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:23,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:25,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:27,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:29,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:31,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:32,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:15:32,282 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:15:33,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:35,541 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:15:35,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:37,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:39,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:41,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:43,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:45,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:47,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:47,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:15:47,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:15:49,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:51,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:53,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:55,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:57,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:15:59,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:01,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:16:03,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:16:03,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:05,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:05,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:16:07,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:09,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:11,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:13,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:15,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:17,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:18,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:16:18,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:16:19,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:21,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:23,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:25,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:27,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:29,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:31,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:16:33,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:16:33,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:35,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:35,689 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:16:37,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:40,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:42,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:44,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:46,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:48,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:48,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:16:48,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:16:50,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:52,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:54,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:56,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:16:58,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:00,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:02,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:04,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:17:04,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:17:04,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:05,764 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:17:06,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:08,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:10,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:12,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:14,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:16,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:18,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:17:19,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:17:20,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:22,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:24,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:26,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:28,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:30,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:32,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:34,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:17:34,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:17:35,841 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:17:36,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:38,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:40,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:42,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:44,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:46,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:48,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:49,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:17:49,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:17:51,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:53,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:55,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:57,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:17:59,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:01,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:03,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:05,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:18:05,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:18:05,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:05,918 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:18:07,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:09,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:11,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:13,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:15,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:17,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:19,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:18:20,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:18:21,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:23,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:25,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:27,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:29,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:31,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:33,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:35,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:35,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:18:35,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:18:35,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:18:37,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:39,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:41,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:43,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:45,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:47,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:49,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:51,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:18:51,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:18:51,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:53,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:55,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:57,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:18:59,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:01,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:03,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:05,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:06,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:19:06,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:19:06,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:19:07,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:09,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:11,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:13,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:15,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:17,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:19,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:19:21,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:19:21,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:23,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:25,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:27,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:30,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:32,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:34,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:36,128 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:19:36,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:19:36,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:19:38,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:40,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:42,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:44,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:46,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:48,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:50,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:52,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:19:52,079 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:19:52,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:54,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:56,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:19:58,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:00,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:02,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:04,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:06,200 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:20:06,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:20:07,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:20:08,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:10,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:12,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:14,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:16,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:18,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:20,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:22,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:22,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:20:22,813 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:20:24,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:26,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:28,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:30,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:32,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:34,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:36,270 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:20:36,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:38,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:20:38,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:20:38,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:40,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:42,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:44,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:46,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:48,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:50,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:52,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:53,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:20:53,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:20:54,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:56,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:20:58,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:00,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:02,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:04,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:06,349 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:21:06,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:08,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:21:08,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:21:08,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:10,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:12,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:14,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:16,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:18,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:20,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:22,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:21:23,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:21:25,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:27,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:29,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:31,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:33,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:35,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:36,425 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:21:37,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:39,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:21:39,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:21:39,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:41,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:43,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:45,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:47,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:49,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:51,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:53,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:54,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:21:54,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:21:55,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:57,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:21:59,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:01,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:03,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:05,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:06,500 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:22:07,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:22:09,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:22:09,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:11,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:13,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:15,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:17,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:19,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:21,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:24,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:24,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:22:24,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:22:26,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:28,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:30,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:32,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:34,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:36,580 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:22:36,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:38,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:40,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:22:40,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:22:40,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:42,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:44,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:46,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:48,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:50,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:52,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:54,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:55,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:22:55,414 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:22:56,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:22:58,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:00,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:02,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:04,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:06,658 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:23:06,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:08,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:10,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:23:10,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:23:10,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:12,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:14,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:16,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:18,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:20,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:22,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:24,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:23:26,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:23:26,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:28,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:30,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:32,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:34,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:36,733 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:23:36,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:38,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:40,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:41,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:23:41,255 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:23:42,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:44,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:46,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:48,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:50,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:52,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:54,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:56,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:23:56,604 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:23:56,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:23:58,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:01,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:03,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:05,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:06,812 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:24:07,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:09,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:11,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:11,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:24:11,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:24:13,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:15,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:17,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:19,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:21,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:23,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:25,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:27,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:24:27,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:24:27,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:29,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:31,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:33,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:35,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:36,888 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:24:37,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:39,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:41,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:42,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:24:42,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:24:43,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:45,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:47,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:49,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:51,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:53,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:55,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:24:57,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:24:57,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:24:59,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:01,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:03,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:05,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:06,964 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:25:07,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:09,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:11,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:13,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:25:13,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:25:13,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:15,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:17,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:19,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:21,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:23,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:25,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:27,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:28,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:25:28,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:25:29,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:31,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:34,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:36,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:37,039 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:25:38,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:40,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:42,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:43,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:25:43,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:25:44,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:46,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:48,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:50,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:52,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:54,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:56,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:25:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:25:58,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:25:58,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:00,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:02,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:04,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:06,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:07,113 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:26:08,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:10,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:12,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:13,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:26:13,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:26:14,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:16,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:18,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:20,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:22,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:24,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:26,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:28,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:29,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:26:29,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:26:30,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:32,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:34,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:36,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:37,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:26:38,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:40,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:42,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:44,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:26:44,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:26:44,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:46,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:48,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:50,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:52,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:54,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:56,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:58,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:26:59,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:26:59,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:27:00,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:02,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:04,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:06,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:07,250 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:27:08,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:10,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:12,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:14,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:15,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:27:15,012 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:27:16,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:18,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:20,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:22,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:24,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:26,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:28,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:27:30,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:27:30,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:32,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:34,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:36,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:37,311 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:27:38,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:40,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:42,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:45,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:27:45,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:27:45,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:47,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:49,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:51,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:53,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:55,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:57,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:27:59,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:00,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:28:00,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:28:01,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:03,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:05,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:07,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:28:07,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:09,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:11,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:13,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:15,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:28:16,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:28:17,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:19,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:21,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:23,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:25,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:27,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:29,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:31,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:28:31,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:28:31,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:33,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:35,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:37,439 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:28:37,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:39,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:41,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:43,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:45,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:46,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:28:46,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:28:47,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:49,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:51,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:53,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:55,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:57,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:28:59,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:01,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:29:01,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:29:01,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:03,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:05,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:07,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:29:07,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:09,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:11,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:13,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:15,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:17,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:29:17,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:29:17,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:19,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:21,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:23,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:25,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:27,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:29,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:31,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:32,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:29:32,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:29:33,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:35,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:37,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:29:37,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:39,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:41,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:43,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:45,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:29:47,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:29:48,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:50,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:52,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:54,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:56,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:29:58,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:00,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:02,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:03,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:30:03,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:30:04,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:06,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:07,622 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:30:08,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:10,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:12,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:14,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:16,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:18,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:30:18,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:30:18,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:20,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:22,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:24,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:26,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:28,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:30,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:32,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:33,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:30:33,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:30:34,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:36,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:37,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:30:38,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:40,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:42,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:44,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:46,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:48,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:48,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:30:48,977 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:30:50,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:53,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:55,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:57,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:30:59,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:01,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:03,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:04,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:31:04,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:31:05,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:07,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:31:07,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:09,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:11,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:13,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:15,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:17,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:31:19,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:31:19,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:21,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:23,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:25,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:27,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:29,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:31,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:33,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:34,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:31:34,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:31:35,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:37,832 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:31:37,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:39,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:41,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:43,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:45,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:47,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:49,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:50,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:31:50,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:31:51,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:53,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:55,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:57,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:31:59,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:01,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:04,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:32:05,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:32:06,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:07,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:32:08,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:10,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:12,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:14,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:16,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:18,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:20,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:32:20,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:32:20,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:22,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:24,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:26,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:28,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:30,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:32,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:34,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:32:35,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:32:36,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:37,986 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:32:38,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:40,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:42,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:44,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:46,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:48,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:50,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:51,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:32:51,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:32:52,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:54,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:56,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:32:58,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:00,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:02,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:04,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:06,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:33:06,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:33:06,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:08,065 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:33:08,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:10,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:12,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:14,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:16,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:19,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:21,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:21,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:33:21,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:33:23,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:25,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:27,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:29,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:31,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:33,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:35,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:36,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:33:36,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:33:37,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:38,142 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:33:39,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:41,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:43,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:45,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:47,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:49,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:51,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:52,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:33:52,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:33:53,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:55,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:57,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:33:59,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:01,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:03,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:06,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:07,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:34:07,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:34:08,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:08,218 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:34:10,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:12,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:14,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:16,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:18,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:20,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:22,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:23,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:34:23,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:34:24,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:26,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:28,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:30,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:32,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:34,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:36,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:38,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:38,297 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:34:38,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:34:38,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:34:40,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:42,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:44,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:46,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:48,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:50,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:52,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:53,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:34:53,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:34:54,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:56,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:34:58,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:00,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:02,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:04,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:06,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:08,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:08,375 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:35:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:35:09,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:35:10,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:12,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:14,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:16,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:18,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:20,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:22,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:24,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:24,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:35:24,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:35:26,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:28,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:31,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:33,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:35,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:37,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:38,448 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:35:39,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:39,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:35:39,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:35:41,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:43,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:45,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:47,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:49,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:51,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:53,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:54,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:35:54,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:35:55,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:57,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:35:59,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:01,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:03,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:05,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:07,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:08,519 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:36:09,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:10,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:36:10,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:36:11,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:13,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:15,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:17,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:19,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:21,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:23,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:25,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:36:25,294 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:36:27,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:29,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:31,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:33,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:35,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:37,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:38,594 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:36:39,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:40,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:36:40,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:36:41,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:43,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:45,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:47,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:49,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:51,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:53,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:55,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:36:55,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:36:55,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:36:57,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:00,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:02,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:04,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:06,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:08,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:08,667 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:37:10,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:11,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:37:11,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:37:12,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:14,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:16,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:18,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:20,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:22,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:24,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:26,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:37:26,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:37:28,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:30,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:32,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:34,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:36,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:38,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:38,737 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:37:40,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:41,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:37:41,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:37:42,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:44,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:46,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:48,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:50,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:52,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:54,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:56,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:37:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:37:57,006 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:37:58,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:00,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:02,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:04,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:06,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:08,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:08,811 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:38:10,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:12,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:12,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:38:12,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:38:14,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:16,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:19,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:21,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:23,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:25,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:27,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:27,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:38:27,540 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:38:29,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:31,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:33,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:35,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:37,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:38,887 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:38:39,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:41,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:42,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:38:42,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:38:43,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:45,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:47,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:49,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:51,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:53,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:55,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:57,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:38:58,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:38:58,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:38:59,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:01,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:03,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:05,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:07,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:08,960 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:39:09,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:11,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:13,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:13,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:39:13,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:39:15,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:17,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:19,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:21,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:23,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:25,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:27,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:29,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:39:29,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:39:29,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:31,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:33,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:35,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:37,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:39,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:39:39,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:42,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:44,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:44,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:39:44,262 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:39:46,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:48,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:50,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:52,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:54,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:56,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:58,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:39:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:39:59,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:40:00,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:02,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:04,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:06,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:08,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:09,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:40:10,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:12,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:14,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:40:14,866 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:40:16,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:18,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:20,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:22,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:24,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:26,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:28,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:30,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:40:30,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:40:30,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:32,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:34,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:40:39,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:40:45,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:40:45,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:41:00,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:41:00,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:41:09,252 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:41:15,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:41:15,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:41:31,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:41:31,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:41:33,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:41:33,575 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 21:41:33,628 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 21:41:33,703 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 21:41:34,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 21:41:34,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:41:35,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:41:39,329 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:41:41,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:41:46,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:41:46,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:41:57,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:42:01,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:42:01,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:42:09,406 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:42:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:42:16,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:42:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:42:32,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:42:39,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:42:47,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:42:47,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:43:02,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:43:02,599 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:43:09,560 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:43:17,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:43:17,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:43:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:43:33,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:43:39,637 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:43:48,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:43:48,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:44:03,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:44:03,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:44:09,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:44:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:44:18,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:44:33,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:44:33,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:44:39,788 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:44:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:44:49,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:45:04,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:45:04,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:45:09,865 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:45:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:45:19,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:45:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:45:34,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:45:39,942 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:45:50,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:45:50,204 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:45:51,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:45:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:52,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:53,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:45:55,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:45:57,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:57,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:45:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:45:59,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:02,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:03,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:05,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:46:05,456 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:46:07,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:08,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:09,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:10,017 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:46:11,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:13,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:15,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:17,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:19,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:20,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:46:20,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:46:20,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:20,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:21,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:23,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:25,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:27,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:29,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:31,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:33,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:33,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:33,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:35,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:46:35,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:46:37,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:38,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:38,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:39,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:40,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:46:41,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:44,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:44,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:46,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:46:49,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:46:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:46:51,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:46:55,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:00,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:02,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:02,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:04,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:06,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:47:06,617 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:47:07,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:07,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:10,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:47:10,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:14,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:16,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:17,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:20,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:21,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:47:21,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:47:22,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:25,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:26,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:26,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:28,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:31,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:31,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:32,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:33,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:34,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:36,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:36,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:37,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:47:37,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:47:37,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:37,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:38,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:39,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:40,243 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:47:40,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:41,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:42,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:42,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:44,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:47,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:47,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:48,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:50,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:51,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:47:52,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:47:52,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:54,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:54,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:56,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:57,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:47:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:47:58,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:00,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:02,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:02,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:02,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:04,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:06,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:07,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:48:07,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:48:08,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:10,316 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:48:10,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:12,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:14,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:16,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:18,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:18,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:18,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:20,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:22,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:48:22,851 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:48:24,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:26,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:28,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:30,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:31,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:32,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:32,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:34,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:36,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:48:38,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:48:40,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:48:42,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:42,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:45,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:45,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:46,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:48:49,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:53,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:48:53,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:48:55,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:48:57,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:01,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:04,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:05,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:06,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:49:08,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:49:09,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:10,461 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:49:10,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:11,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:12,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:15,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:17,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:18,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:21,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:23,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:23,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:49:23,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:49:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:27,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:28,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:29,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:30,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:30,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:32,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:35,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:37,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:37,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:37,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:38,363 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 21:49:38,575 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 21:49:38,637 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 21:49:39,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:49:39,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:49:39,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:39,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 21:49:39,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:40,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:49:41,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:42,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:43,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:45,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:46,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:46,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:47,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:48,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:48,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:49,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:51,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:52,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:53,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:54,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:49:54,367 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:49:55,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:55,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:56,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:49:57,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:49:59,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:00,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:01,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:03,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:03,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:03,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:05,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:07,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:09,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:09,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:50:09,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:50:10,720 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:50:11,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:13,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:15,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:17,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:17,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:18,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:19,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:21,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:23,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:24,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:50:24,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:50:25,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:27,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:28,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:29,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:31,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:39,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:40,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:50:40,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:50:40,794 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:50:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:42,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:44,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:45,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:50:48,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:52,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:50:55,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:50:55,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:50:56,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:00,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:04,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:06,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:09,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:10,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:51:10,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:51:10,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:10,954 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:51:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:12,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:13,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:13,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:16,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:18,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:19,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:22,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:24,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:51:25,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:51:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:26,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:28,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:32,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:34,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:34,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:35,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:36,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:37,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:38,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:39,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:39,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:40,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:40,942 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:51:41,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:51:41,304 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:51:42,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:43,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:43,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:44,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:46,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:46,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:48,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:49,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:50,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:50,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:51,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:52,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:54,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:56,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:51:56,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:51:56,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:51:57,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:57,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:51:58,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:00,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:02,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:04,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:04,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:04,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:06,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:08,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:10,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:11,019 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:52:11,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:52:11,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:52:12,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:12,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:14,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:15,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:15,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:16,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:18,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:20,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:23,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:27,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:52:27,037 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:52:31,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:31,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:33,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:38,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:39,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:41,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:52:42,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:52:42,262 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:52:43,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:43,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:45,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:45,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:47,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:48,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:48,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:52:51,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:55,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:52:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:52:57,491 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:52:57,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:01,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:04,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:05,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:07,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:11,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:53:11,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:12,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:53:12,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:53:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:13,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:18,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:19,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:21,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:23,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:25,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:25,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:26,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:26,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:27,373 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 21:53:27,588 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 21:53:27,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:27,652 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 21:53:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:53:27,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:53:28,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:28,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 21:53:28,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:29,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:31,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:33,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:35,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:37,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:37,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:38,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:38,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:39,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:41,285 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:53:41,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:42,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:53:43,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:53:43,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:43,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:44,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:45,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:46,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:47,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:48,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:49,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:51,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:53,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:53,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:53:55,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:57,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:53:58,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:53:58,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:53:59,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:01,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:02,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:03,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:05,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:07,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:09,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:11,360 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:54:11,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:13,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:13,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:13,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:54:13,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:54:15,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:17,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:19,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:27,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:27,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:54:28,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:54:29,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:35,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:39,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:41,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:41,430 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:54:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:43,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:44,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:54:44,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:54:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:51,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:53,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:55,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:54:57,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:54:59,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:54:59,565 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:55:01,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:03,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:06,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:07,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:10,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:11,503 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:55:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:11,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:13,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:55:14,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:55:16,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:17,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:18,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:20,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:21,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:21,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:22,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:24,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:24,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:25,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:25,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:26,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:27,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:55:30,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:55:30,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:31,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:32,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:33,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:34,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:35,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:35,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:36,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:37,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:39,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:40,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:41,577 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:55:42,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:42,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:43,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:43,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:44,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:45,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:55:45,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:55:46,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:46,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:48,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:49,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:49,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:50,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:51,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:52,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:53,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:53,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:54,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:56,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:55:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:55:58,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:00,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:56:00,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:56:00,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:02,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:04,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:05,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:05,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:06,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:08,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:10,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:10,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:11,654 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:56:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:12,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:13,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:14,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:56:15,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:56:16,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:18,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:26,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:30,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:31,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:56:31,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:56:31,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:31,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:34,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:38,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:41,730 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:56:42,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:45,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:45,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:46,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:56:46,461 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:56:46,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:47,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:47,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:50,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:51,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:54,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:56,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:56:56,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:56:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:00,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:01,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:57:01,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:57:02,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:05,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:05,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:07,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:09,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:11,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:11,806 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:57:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:12,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:13,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:15,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:16,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:16,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:57:16,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:57:17,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:17,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:18,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:19,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:20,300 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 21:57:20,517 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 21:57:20,581 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 21:57:20,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 21:57:20,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:21,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:22,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:23,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:25,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:25,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:27,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:28,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:29,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:30,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:30,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:31,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:32,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:57:32,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:57:33,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:33,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:34,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:35,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:37,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:37,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:39,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:41,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:41,945 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:57:42,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:43,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:45,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:46,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:57:47,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:57:47,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:49,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:49,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:51,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:53,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:54,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:57:55,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:57,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:57:59,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:01,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:01,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:02,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:58:02,590 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:58:03,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:05,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:05,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:06,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:06,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:07,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:09,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:09,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:11,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:12,014 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:58:17,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:58:17,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:58:21,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:27,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:31,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:58:33,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:58:35,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:35,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:37,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:41,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:42,089 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:58:44,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:44,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:45,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:47,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:48,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:58:48,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:58:49,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:51,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:52,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:53,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:56,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:58:57,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:58:59,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:03,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:59:03,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:59:03,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:06,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:06,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:08,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:12,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:59:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:12,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:13,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:13,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:13,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:14,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:15,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:16,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:18,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:18,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:59:18,918 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:59:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:19,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:20,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:22,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:22,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:24,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:24,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:25,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:26,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:27,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:28,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:28,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:30,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:32,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:32,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:33,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:59:34,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:59:34,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:34,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:36,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:36,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:36,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:38,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:39,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:39,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:40,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:42,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,243 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 21:59:42,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:44,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:45,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:46,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:48,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:49,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 21:59:49,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 21:59:50,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:50,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:51,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:51,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:52,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:54,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:54,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:56,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 21:59:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 21:59:58,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:00,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:01,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:01,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:02,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:04,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:00:04,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:00:04,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:06,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:12,321 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:00:16,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:00:19,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:00:20,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:24,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:24,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:28,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:32,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:35,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:00:35,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:00:36,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:40,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:42,396 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:00:42,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:44,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:46,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:48,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:50,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:00:50,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:00:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:51,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:52,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:54,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:00:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:55,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:00:58,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:00,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:02,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:04,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:01:05,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:01:08,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:08,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:09,032 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 22:01:09,231 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 22:01:09,289 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 22:01:09,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 22:01:09,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:10,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:11,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:11,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:12,499 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:01:12,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:13,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:13,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:14,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:16,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:17,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:18,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:19,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:20,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:21,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:01:21,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:01:24,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:25,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:26,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:28,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:28,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:29,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:29,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:30,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:30,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:31,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:32,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:33,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:34,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:36,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:01:36,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:01:36,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:36,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:36,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:37,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:38,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:39,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:40,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:41,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:42,573 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:01:43,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:45,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:45,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:47,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:49,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:51,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:01:51,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:01:51,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:53,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:53,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:55,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:01:57,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:01:59,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:01,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:06,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:02:06,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:02:09,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:09,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:12,649 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:02:15,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:19,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:22,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:02:22,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:02:22,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:23,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:27,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:29,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:33,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:37,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:02:37,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:02:37,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:39,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:42,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:02:44,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:46,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:50,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:52,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:02:52,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:02:52,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:54,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:55,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:57,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:58,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:02:59,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:02:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:00,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:01,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:02,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:06,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:03:08,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:03:08,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:10,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:10,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:11,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:12,808 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:03:12,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:14,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:14,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:15,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:15,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:16,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:18,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:19,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:19,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:20,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:21,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:21,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:03:23,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:03:24,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:25,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:25,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:26,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:28,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:29,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:30,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:31,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:32,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:33,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:34,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:35,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:35,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:36,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:37,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:37,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:03:38,578 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:03:38,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:38,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:40,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:41,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:42,884 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:03:42,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:43,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:44,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:45,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:45,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:46,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:48,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:49,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:50,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:52,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:53,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:03:53,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:03:54,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:03:54,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:03:56,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:06,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:09,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:04:09,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:04:10,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:12,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:04:14,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:18,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:22,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:23,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:24,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:04:24,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:04:26,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:28,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:32,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:36,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:38,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:39,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:04:39,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:04:42,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:43,034 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:04:44,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:48,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:49,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:49,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:50,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:51,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:51,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:54,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:55,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:04:55,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:04:56,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:57,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:57,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:58,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:04:59,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:04:59,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:05:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:05:00,148 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 22:05:00,369 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 22:05:00,423 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 22:05:00,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 22:05:00,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:02,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:04,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:06,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:08,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:05:10,401 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:05:10,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:12,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:13,160 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:05:14,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:17,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:19,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:21,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:23,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:25,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:05:25,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:05:25,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:27,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:29,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:31,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:33,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:35,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:37,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:39,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:40,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:05:40,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:05:41,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:43,237 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:05:43,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:45,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:47,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:49,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:51,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:53,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:55,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:05:56,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:05:57,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:05:59,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:01,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:03,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:05,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:07,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:09,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:11,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:06:11,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:06:11,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:13,311 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:06:13,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:15,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:17,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:19,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:21,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:24,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:26,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:06:26,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:06:26,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:28,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:30,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:32,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:34,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:36,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:38,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:40,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:42,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:06:42,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:06:42,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:43,386 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:06:44,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:46,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:48,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:50,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:52,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:54,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:56,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:06:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:06:57,329 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:06:59,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:01,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:03,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:05,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:07,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:09,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:11,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:12,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:07:12,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:07:13,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:13,463 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:07:15,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:17,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:19,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:21,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:23,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:25,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:27,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:27,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:07:27,878 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:07:29,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:31,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:33,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:35,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:37,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:39,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:41,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:43,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:07:43,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:07:43,536 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:07:44,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:46,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:48,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:50,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:52,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:54,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:56,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:58,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:07:58,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:07:58,421 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:08:00,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:02,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:04,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:06,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:08,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:10,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:12,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:13,611 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:08:13,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:08:13,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:08:14,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:16,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:18,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:20,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:22,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:24,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:26,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:28,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:28,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:08:28,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:08:30,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:32,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:34,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:36,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:38,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:40,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:42,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:43,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:08:44,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:44,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:08:44,099 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:08:46,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:48,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:50,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:52,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:54,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:56,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:58,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:08:59,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:08:59,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:09:00,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:02,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:04,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:06,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:08,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:10,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:12,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:13,763 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:09:14,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:14,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:09:14,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:09:16,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:18,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:20,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:22,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:24,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:26,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:29,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:29,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:09:29,856 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:09:31,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:33,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:35,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:37,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:39,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:41,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:43,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:43,840 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:09:45,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:09:45,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:09:47,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:49,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:51,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:53,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:55,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:57,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:09:59,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:10:00,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:10:01,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:03,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:05,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:07,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:09,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:11,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:13,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:13,915 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:10:15,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:15,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:10:15,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:10:17,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:19,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:21,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:23,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:25,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:27,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:29,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:30,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:10:30,993 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:10:31,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:33,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:35,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:37,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:39,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:41,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:43,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:43,991 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:10:45,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:46,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:10:46,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:10:47,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:49,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:51,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:53,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:55,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:10:58,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:00,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:11:01,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:11:02,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:04,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:06,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:08,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:10,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:12,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:14,055 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:11:14,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:16,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:16,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:11:16,696 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:11:18,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:20,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:22,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:24,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:26,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:28,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:30,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:11:31,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:11:32,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:34,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:36,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:38,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:40,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:42,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:44,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:44,131 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:11:46,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:47,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:11:47,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:11:48,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:50,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:52,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:54,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:56,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:11:58,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:00,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:02,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:12:02,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:12:04,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:06,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:09,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:11,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:13,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:14,189 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:12:15,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:17,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:18,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:12:18,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:12:19,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:21,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:23,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:25,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:27,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:29,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:31,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:33,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:33,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:12:33,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:12:35,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:37,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:39,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:41,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:43,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:44,254 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:12:45,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:47,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:48,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:12:48,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:12:49,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:51,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:53,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:55,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:57,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:12:59,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:01,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:03,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:04,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:13:04,204 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:13:05,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:07,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:09,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:11,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:13,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:14,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:13:15,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:17,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:19,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:19,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:13:19,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:13:21,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:23,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:26,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:28,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:30,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:32,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:34,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:13:34,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:13:36,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:38,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:40,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:42,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:44,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:44,377 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:13:46,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:48,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:49,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:13:49,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:13:50,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:52,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:54,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:56,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:13:58,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:00,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:02,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:04,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:05,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:14:05,262 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:14:06,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:08,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:10,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:12,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:14,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:14,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:14:16,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:18,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:20,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:14:20,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:14:22,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:24,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:26,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:28,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:30,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:32,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:34,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:35,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:14:35,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:14:36,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:38,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:40,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:42,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:44,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:44,522 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:14:46,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:48,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:50,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:50,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:14:50,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:14:52,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:54,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:56,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:14:58,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:00,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:02,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:04,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:06,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:06,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:15:06,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:15:08,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:10,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:12,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:14,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:14,596 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:15:16,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:18,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:20,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:21,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:15:21,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:15:22,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:24,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:26,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:29,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:31,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:33,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:35,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:36,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:15:36,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:15:37,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:39,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:41,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:43,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:44,668 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:15:45,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:47,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:49,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:51,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:52,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:15:52,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:15:53,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:55,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:57,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:15:59,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:01,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:03,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:05,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:07,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:07,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:16:07,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:16:09,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:11,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:13,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:14,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:16:15,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:17,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:19,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:21,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:16:22,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:16:23,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:25,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:27,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:29,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:31,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:33,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:35,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:37,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:37,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:16:37,980 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:16:39,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:41,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:43,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:44,817 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:16:45,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:47,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:49,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:51,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:16:53,206 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:16:53,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:55,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:57,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:16:59,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:01,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:04,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:06,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:08,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:17:08,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:17:10,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:12,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:14,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:14,875 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:17:16,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:18,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:20,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:22,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:17:23,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:17:24,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:26,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:28,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:30,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:32,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:34,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:36,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:38,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:17:38,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:17:40,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:42,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:44,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:44,948 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:17:46,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:48,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:50,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:52,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:54,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:54,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:17:54,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:17:56,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:17:58,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:00,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:02,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:04,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:06,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:08,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:18:09,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:18:10,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:12,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:14,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:15,023 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:18:16,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:18,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:20,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:22,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:24,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:24,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:18:24,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:18:26,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:28,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:30,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:32,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:34,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:36,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:38,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:18:40,091 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:18:40,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:42,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:44,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:45,101 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:18:46,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:49,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:51,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:53,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:55,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:55,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:18:55,324 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:18:57,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:18:59,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:01,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:03,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:05,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:07,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:09,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:19:10,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:19:11,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:13,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:15,164 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:19:15,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:17,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:19,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:21,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:23,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:25,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:26,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:19:26,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:19:27,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:29,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:31,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:33,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:35,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:37,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:39,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:41,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:41,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:19:41,376 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:19:43,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:45,239 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:19:45,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:47,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:49,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:51,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:53,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:55,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:19:56,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:19:57,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:19:59,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:01,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:03,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:05,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:07,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:09,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:11,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:20:11,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:20:13,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:15,315 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:20:15,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:17,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:19,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:21,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:23,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:25,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:27,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:20:27,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:20:27,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:29,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:31,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:33,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:35,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:38,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:40,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:42,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:42,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:20:42,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:20:44,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:45,379 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:20:46,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:48,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:50,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:52,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:54,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:56,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:20:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:20:57,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:20:58,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:00,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:02,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:04,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:06,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:08,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:10,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:12,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:21:13,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:21:14,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:15,450 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:21:16,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:18,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:20,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:22,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:24,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:26,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:28,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:21:28,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:21:30,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:32,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:34,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:36,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:38,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:40,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:42,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:43,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:21:43,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:21:44,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:45,514 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:21:46,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:48,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:50,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:52,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:54,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:56,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:58,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:21:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:21:58,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:22:00,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:02,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:04,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:06,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:08,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:10,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:12,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:22:14,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:22:14,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:15,580 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:22:16,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:18,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:20,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:22,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:24,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:26,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:28,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:22:29,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:22:30,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:32,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:34,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:36,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:38,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:40,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:43,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:44,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:22:44,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:22:45,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:45,656 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:22:47,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:49,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:51,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:53,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:55,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:57,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:59,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:22:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:22:59,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:23:01,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:03,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:05,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:07,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:09,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:11,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:13,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:15,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:23:15,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:23:15,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:15,734 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:23:17,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:19,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:21,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:23,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:25,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:27,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:29,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:30,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:23:30,600 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:23:31,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:33,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:35,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:37,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:39,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:41,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:43,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:45,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:45,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:23:45,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:23:45,826 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:23:47,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:49,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:51,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:53,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:55,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:57,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:23:59,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:01,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:24:01,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:24:01,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:03,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:05,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:07,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:09,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:11,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:13,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:15,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:15,889 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:24:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:24:16,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:24:17,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:19,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:21,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:23,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:26,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:28,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:30,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:24:31,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:24:32,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:34,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:36,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:38,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:40,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:42,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:44,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:45,960 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:24:46,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:24:47,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:24:48,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:50,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:52,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:54,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:56,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:24:58,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:00,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:02,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:25:02,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:25:02,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:04,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:06,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:08,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:10,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:12,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:14,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:16,040 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:25:16,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:17,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:25:17,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:25:18,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:20,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:22,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:24,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:26,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:28,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:30,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:32,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:25:32,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:25:34,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:36,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:38,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:40,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:42,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:44,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:46,120 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:25:46,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:48,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:25:48,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:25:48,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:50,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:53,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:55,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:57,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:25:59,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:01,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:03,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:26:03,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:26:03,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:05,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:07,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:09,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:11,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:13,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:15,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:16,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:26:17,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:18,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:26:18,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:26:19,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:21,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:23,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:25,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:27,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:29,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:31,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:33,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:26:34,063 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:26:35,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:37,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:39,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:41,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:43,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:45,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:46,272 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:26:47,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:49,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:26:49,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:26:49,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:51,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:53,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:55,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:57,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:26:59,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:01,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:03,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:27:04,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:27:05,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:07,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:09,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:11,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:13,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:15,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:16,347 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:27:17,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:19,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:19,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:27:19,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:27:21,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:23,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:25,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:28,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:30,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:32,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:34,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:35,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:27:35,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:27:36,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:38,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:40,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:42,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:44,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:46,422 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:27:46,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:48,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:50,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:27:50,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:27:50,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:52,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:54,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:56,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:27:58,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:00,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:02,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:04,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:05,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:28:05,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:28:06,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:08,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:10,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:12,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:14,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:16,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:28:16,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:18,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:20,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:20,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:28:20,977 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:28:22,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:24,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:26,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:28,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:30,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:32,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:34,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:36,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:28:36,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:28:36,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:38,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:40,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:42,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:44,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:46,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:46,567 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:28:48,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:50,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:28:51,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:28:52,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:55,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:57,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:28:59,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:01,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:03,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:05,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:29:06,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:29:07,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:09,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:11,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:13,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:15,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:16,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:29:17,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:19,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:21,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:29:21,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:29:23,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:25,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:27,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:29,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:31,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:33,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:35,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:29:37,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:29:37,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:39,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:41,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:43,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:45,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:46,676 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:29:47,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:49,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:51,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:52,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:29:52,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:29:53,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:55,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:57,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:29:59,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:01,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:03,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:05,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:07,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:30:07,788 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:30:09,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:11,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:13,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:15,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:16,728 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:30:17,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:20,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:22,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:30:23,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:30:24,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:26,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:28,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:30,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:32,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:34,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:36,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:38,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:30:38,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:30:38,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:40,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:42,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:44,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:46,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:46,801 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:30:48,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:50,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:52,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:30:53,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:30:54,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:56,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:30:58,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:00,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:02,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:04,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:06,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:08,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:08,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:31:08,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:31:10,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:12,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:14,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:16,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:16,873 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:31:18,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:20,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:22,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:24,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:31:24,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:31:24,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:26,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:28,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:30,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:32,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:34,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:36,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:38,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:31:39,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:31:40,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:42,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:44,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:46,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:46,947 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:31:48,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:50,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:52,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:54,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:54,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:31:54,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:31:56,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:31:59,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:01,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:03,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:05,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:07,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:09,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:10,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:32:10,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:32:11,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:13,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:15,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:17,023 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:32:17,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:19,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:21,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:23,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:32:25,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:32:25,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:27,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:29,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:31,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:33,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:35,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:37,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:39,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:32:40,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:32:41,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:43,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:45,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:47,095 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:32:47,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:49,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:51,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:53,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:55,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:55,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:32:55,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:32:57,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:32:59,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:01,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:03,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:05,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:07,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:09,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:33:11,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:33:11,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:13,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:15,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:17,174 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:33:17,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:19,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:21,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:23,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:25,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:26,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:33:26,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:33:27,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:29,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:31,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:33,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:35,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:37,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:39,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:41,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:33:41,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:33:43,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:45,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:47,247 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:33:47,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:50,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:52,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:54,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:56,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:33:56,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:33:56,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:33:58,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:00,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:02,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:04,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:06,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:08,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:10,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:34:12,323 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:34:12,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:14,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:16,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:17,324 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:34:18,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:20,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:22,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:24,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:26,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:27,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:34:27,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:34:28,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:30,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:32,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:34,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:36,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:38,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:40,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:42,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:43,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:34:43,022 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:34:44,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:46,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:47,400 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:34:48,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:50,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:52,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:54,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:57,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:34:58,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:34:58,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:34:59,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:01,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:03,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:05,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:07,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:09,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:11,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:13,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:35:13,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:35:13,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:15,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:17,476 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:35:17,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:19,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:21,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:23,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:25,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:27,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:28,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:35:28,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:35:29,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:31,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:33,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:35,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:37,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:39,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:41,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:43,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:44,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:35:44,094 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:35:45,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:47,550 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:35:47,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:49,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:51,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:53,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:55,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:57,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:35:59,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:35:59,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:35:59,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:01,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:03,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:05,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:07,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:09,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:11,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:13,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:36:14,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:36:15,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:17,612 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:36:17,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:19,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:21,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:23,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:25,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:27,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:29,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:29,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:36:29,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:36:31,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:33,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:35,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:37,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:39,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:41,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:43,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:36:45,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:36:45,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:47,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:36:47,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:49,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:51,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:54,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:56,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:36:58,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:00,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:37:00,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:37:00,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:02,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:04,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:06,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:08,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:10,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:12,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:14,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:15,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:37:15,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:37:16,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:17,765 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:37:18,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:20,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:22,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:24,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:26,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:28,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:30,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:31,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:37:31,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:37:32,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:34,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:36,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:38,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:40,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:42,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:44,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:46,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:37:46,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:37:46,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:47,840 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:37:48,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:50,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:52,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:54,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:56,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:37:58,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:00,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:01,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:38:01,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:38:02,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:04,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:06,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:08,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:10,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:12,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:14,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:16,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:16,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:38:16,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:38:17,916 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:38:18,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:20,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:22,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:25,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:27,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:29,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:31,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:32,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:38:32,141 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:38:33,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:35,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:37,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:39,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:41,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:43,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:45,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:47,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:38:47,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:38:47,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:47,992 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:38:49,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:51,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:53,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:55,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:57,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:38:59,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:01,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:02,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:39:02,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:39:03,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:05,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:07,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:09,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:11,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:13,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:15,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:17,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:17,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:39:17,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:39:18,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:39:19,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:21,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:23,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:25,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:27,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:29,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:31,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:33,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:39:33,250 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:39:33,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:35,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:37,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:39,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:41,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:43,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:45,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:47,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:48,147 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:39:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:39:48,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:39:49,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:51,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:53,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:55,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:57,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:39:59,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:01,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:03,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:40:03,748 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:40:03,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:05,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:07,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:09,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:11,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:13,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:15,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:18,223 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:40:18,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:40:19,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:40:20,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:22,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:24,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:26,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:28,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:30,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:32,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:40:34,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:40:34,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:36,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:38,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:40,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:42,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:44,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:46,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:48,302 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:40:48,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:49,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:40:49,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:40:50,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:52,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:54,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:56,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:40:58,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:00,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:02,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:04,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:05,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:41:05,035 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:41:06,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:08,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:10,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:12,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:14,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:16,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:18,383 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:41:18,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:20,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:41:20,258 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:41:20,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:22,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:24,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:26,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:28,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:30,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:32,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:34,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:35,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:41:35,597 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:41:36,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:39,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:41,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:43,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:45,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:47,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:48,459 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:41:49,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:41:50,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:41:51,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:53,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:55,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:57,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:41:59,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:01,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:03,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:05,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:06,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:42:06,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:42:07,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:09,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:11,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:13,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:15,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:17,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:18,531 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:42:19,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:21,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:42:21,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:42:21,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:23,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:25,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:27,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:29,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:31,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:33,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:35,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:36,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:42:36,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:42:37,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:39,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:41,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:43,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:45,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:47,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:48,605 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:42:49,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:51,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:51,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:42:51,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:42:53,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:55,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:57,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:42:59,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:01,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:04,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:06,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:07,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:43:07,262 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:43:08,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:10,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:12,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:14,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:16,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:18,677 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:43:18,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:20,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:43:22,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:43:22,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:24,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:26,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:28,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:30,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:32,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:34,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:36,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:37,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:43:37,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:43:38,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:40,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:42,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:44,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:46,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:48,751 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:43:48,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:50,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:52,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:53,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:43:53,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:43:54,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:56,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:43:58,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:00,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:02,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:04,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:06,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:44:08,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:44:08,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:10,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:12,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:14,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:16,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:18,824 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:44:18,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:20,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:22,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:23,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:44:23,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:44:24,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:26,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:28,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:30,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:32,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:34,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:36,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:38,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:44:38,977 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:44:41,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:43,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:45,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:47,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:48,894 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:44:49,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:51,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:53,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:44:54,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:44:55,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:57,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:44:59,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:01,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:03,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:05,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:07,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:45:09,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:45:09,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:11,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:13,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:15,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:17,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:18,970 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:45:19,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:21,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:23,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:45:24,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:45:25,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:27,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:29,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:31,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:33,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:35,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:37,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:39,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:40,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:45:40,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:45:41,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:43,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:45,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:47,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:49,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:45:49,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:51,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:53,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:55,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:45:55,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:45:56,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:45:58,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:00,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:02,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:04,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:06,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:08,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:46:10,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:46:10,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:12,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:14,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:16,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:18,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:19,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:46:20,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:22,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:24,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:26,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:46:26,126 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:46:26,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:28,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:30,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:32,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:34,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:36,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:38,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:40,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:41,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:46:41,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:46:42,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:44,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:46,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:48,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:49,194 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:46:50,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:52,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:54,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:46:56,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:46:56,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:47:11,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:47:11,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:47:19,270 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:47:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:47:27,141 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:47:42,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:47:42,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:47:49,343 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:47:52,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:47:52,964 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 22:47:53,014 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 22:47:53,091 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 22:47:54,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 22:47:54,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:47:55,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:47:56,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:47:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:47:57,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:48:00,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:48:12,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:48:12,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:48:18,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:48:19,426 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:48:28,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:48:28,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:48:43,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:48:43,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:48:49,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:48:58,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:48:58,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:49:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:49:13,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:49:19,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:49:29,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:49:29,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:49:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:49:44,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:49:49,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:49:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:49:59,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:50:14,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:50:14,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:50:19,719 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:50:30,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:50:30,020 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:50:45,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:50:45,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:50:49,797 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:51:00,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:51:00,473 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:51:12,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:51:15,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:51:15,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:51:19,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:51:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:51:30,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:51:46,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:51:46,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:51:49,952 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:52:01,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:52:01,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:52:14,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:16,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:16,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:52:16,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:52:20,029 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:52:20,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:21,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:22,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:24,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:25,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:25,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:26,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:27,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:27,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:28,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:30,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:30,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:52:31,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:52:32,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:32,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:34,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:35,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:35,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:36,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:38,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:40,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:41,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:42,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:43,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:44,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:44,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:45,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:46,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:52:47,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:52:47,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:48,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:49,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:49,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:50,103 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:52:50,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:51,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:51,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:52,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:53,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:53,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:54,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:55,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:52:56,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:56,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:52:58,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:00,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:00,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:00,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:01,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:02,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:53:02,387 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:53:04,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:06,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:07,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:07,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:15,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:53:17,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:53:19,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:20,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:53:23,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:27,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:31,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:53:32,959 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:53:35,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:37,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:37,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:38,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:39,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:41,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:45,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:47,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:53:48,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:53:50,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:53:51,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:53,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:57,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:59,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:53:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:53:59,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:01,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:03,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:03,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:54:03,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:54:05,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:05,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:05,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:06,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:06,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:06,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:06,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:09,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:10,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:11,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:15,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:17,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:18,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:54:18,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:54:19,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:20,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:54:21,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:21,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:21,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:22,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:23,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:24,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:24,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:25,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:25,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:27,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:27,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:29,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:29,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:31,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:33,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:34,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:34,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:54:34,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:54:35,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:36,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:36,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:37,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:38,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:39,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:41,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:41,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:43,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:43,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:44,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:45,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:47,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:47,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:49,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:49,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:49,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:54:49,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:54:50,395 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:54:50,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:50,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:51,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:51,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:53,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:55,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:55,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:54:57,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:57,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:54:59,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:03,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:04,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:55:04,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:55:09,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:12,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:12,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:15,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:19,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:55:20,056 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:55:20,467 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:55:23,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:27,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:31,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:35,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:55:35,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:55:37,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:38,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:38,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:40,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:55:42,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:44,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:48,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:50,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:50,535 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:55:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:55:50,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:55:54,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:56,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:55:58,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:00,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:00,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:03,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:04,226 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 22:56:04,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:04,419 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 22:56:04,491 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 22:56:05,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 22:56:05,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:05,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:56:05,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:56:06,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:06,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:08,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:10,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:11,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:12,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:13,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:13,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:14,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:16,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:20,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:20,633 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:56:21,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:56:21,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:56:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:21,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:22,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:24,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:24,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:26,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:26,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:28,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:30,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:31,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:32,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:34,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:34,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:36,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:56:36,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:56:36,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:37,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:37,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:40,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:40,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:42,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:44,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:46,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:47,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:49,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:50,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:50,707 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:56:51,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:56:51,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:56:52,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:52,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:53,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:56:54,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:56:56,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:04,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:04,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:06,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:57:06,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:57:08,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:10,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:10,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:12,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:16,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:20,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:20,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:57:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:57:22,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:57:24,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:26,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:30,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:32,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:32,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:32,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:36,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:37,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:37,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:57:37,479 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:57:38,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:42,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:44,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:46,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:50,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:50,861 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:57:52,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:57:52,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:57:53,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:55,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:55,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:57:57,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:57:59,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:01,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:03,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:05,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:07,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:07,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:07,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:58:08,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:58:09,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:09,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:11,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:13,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:15,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:17,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:19,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:19,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:19,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:20,936 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:58:21,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:22,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:22,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:58:23,261 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:58:23,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:24,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:25,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:27,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:29,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:29,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:29,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:31,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:33,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:35,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:36,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:36,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:37,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:58:38,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:58:39,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:39,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:40,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:40,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:41,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:42,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:42,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:43,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:43,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:45,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:47,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:51,011 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:58:53,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:58:53,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:58:55,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:58:56,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:56,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:58:58,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:01,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:04,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:04,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:05,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:59:09,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:59:09,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:13,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:14,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:15,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:19,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:21,087 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:59:23,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:59:24,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:59:25,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:29,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:31,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:32,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:32,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:35,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:36,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:36,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:37,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:39,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:59:39,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:59:43,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:45,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:47,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:48,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:48,858 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 22:59:48,990 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 22:59:49,051 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 22:59:49,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 22:59:49,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:50,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:51,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 22:59:51,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:53,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:54,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 22:59:54,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 22:59:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:55,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 22:59:57,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 22:59:59,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:01,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:01,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:01,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:03,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:04,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:05,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:05,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:07,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:07,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:08,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:09,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:10,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:00:10,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:00:12,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:13,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:13,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:14,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:16,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:17,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:17,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:18,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:19,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:19,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:20,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:21,457 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:00:22,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:22,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:22,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:24,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:26,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:26,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:28,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:28,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:29,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:00:30,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:00:30,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:30,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:31,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:31,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:32,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:33,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:34,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:34,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:37,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:38,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:40,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:43,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:45,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:00:45,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:00:49,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:49,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:50,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:51,320 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:00:54,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:54,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:00:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:00:58,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:00,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:01:00,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:01:02,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:06,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:06,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:06,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:08,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:12,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:13,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:13,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:15,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:01:15,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:01:16,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:16,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:18,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:21,396 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:01:22,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:24,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:26,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:30,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:31,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:01:31,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:01:31,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:32,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:34,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:38,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:40,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:40,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:42,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:44,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:46,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:46,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:01:46,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:01:48,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:50,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:51,472 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:01:52,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:53,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:54,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:54,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:56,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:58,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:01:58,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:01:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:00,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:01,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:02:01,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:02:02,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:03,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:04,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:05,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:06,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:08,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:10,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:10,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:10,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:11,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:12,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:13,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:14,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:15,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:16,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:16,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:02:17,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:02:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:18,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:20,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:21,548 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:02:22,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:22,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:23,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:24,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:26,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:27,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:28,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:29,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:29,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:30,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:32,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:02:32,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:02:32,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:35,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:41,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:41,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:46,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:47,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:47,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:02:47,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:02:48,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:51,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:51,623 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:02:52,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:52,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:54,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:02:55,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:02:59,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:01,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:02,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:03:02,856 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:03:05,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:07,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:09,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:11,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:12,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:13,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:17,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:18,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:03:18,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:03:19,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:20,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:21,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:21,699 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:03:25,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:27,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:29,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:29,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:29,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:31,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:32,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:32,814 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 23:03:33,024 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 23:03:33,104 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 23:03:33,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:33,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:03:33,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:03:33,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 23:03:33,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:34,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:37,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:39,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:41,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:43,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:45,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:47,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:48,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:03:48,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:03:49,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:51,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:51,853 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:03:53,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:53,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:54,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:54,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:55,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:56,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:57,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:03:58,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:03:59,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:00,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:00,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:01,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:03,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:04,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:04:04,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:04:05,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:05,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:07,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:08,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:09,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:10,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:11,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:13,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:14,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:15,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:17,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:04:19,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:04:19,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:19,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:19,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:21,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:21,929 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:04:23,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:23,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:27,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:28,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:29,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:29,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:04:34,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:04:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:34,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:35,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:39,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:39,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:39,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:43,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:47,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:49,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:49,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:04:49,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:04:52,007 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:04:52,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:04:53,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:04:58,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:00,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:04,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:04,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:04,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:05:04,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:05:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:06,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:10,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:12,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:14,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:15,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:18,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:20,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:05:20,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:05:20,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:22,082 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:05:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:22,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:24,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:26,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:30,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:32,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:34,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:35,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:05:35,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:05:36,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:38,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:40,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:42,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:44,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:46,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:47,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:48,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:50,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:05:50,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:05:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,155 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:05:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:52,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:53,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:53,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:54,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:56,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:58,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:05:58,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:05:59,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:00,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:02,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:03,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:04,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:06:05,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:06:06,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:07,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:07,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:08,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:09,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:10,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:11,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:12,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:14,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:15,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:15,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:17,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:17,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:17,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:17,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:20,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:21,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:06:21,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:06:22,228 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:06:22,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:25,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:28,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:29,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:29,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:32,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:36,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:36,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:06:36,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:06:38,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:40,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:44,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:45,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:45,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:47,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:48,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:50,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:06:52,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:06:52,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,407 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:06:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:52,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:54,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:56,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:06:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:06:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:00,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:02,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:06,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:07,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:07:07,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:07:08,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:12,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:14,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:16,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:17,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:17,886 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 23:07:18,082 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 23:07:18,180 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 23:07:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:18,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 23:07:18,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:19,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:22,402 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:07:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:07:22,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:07:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:23,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:25,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:27,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:29,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:31,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:33,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:35,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:37,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:07:37,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:07:39,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:41,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:43,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:45,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:45,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:47,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:47,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:48,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:49,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:51,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:51,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:52,499 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:07:53,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:07:53,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:07:53,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:53,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:55,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:07:59,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:07:59,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:01,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:01,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:02,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:03,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:05,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:06,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:07,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:08:08,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:08:09,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:11,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:15,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:17,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:20,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,579 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:08:22,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:22,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:23,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:23,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:08:23,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:08:27,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:27,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:31,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:32,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:35,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:37,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:39,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:08:39,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:08:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:40,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:41,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:42,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:43,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:47,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:48,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:48,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:49,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:52,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:52,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:08:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:53,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:08:54,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:08:55,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:57,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:08:59,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:08:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:03,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:05,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:07,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:07,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:08,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:09:09,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:09:09,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:10,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:10,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:11,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:13,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:16,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:17,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:19,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:21,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:22,731 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:09:23,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:09:24,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:09:25,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:27,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:29,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:31,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:33,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:35,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:37,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:39,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:40,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:09:40,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:09:40,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:40,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:41,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:43,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:44,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:45,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:45,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:47,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:49,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:50,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:51,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:52,808 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:09:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:53,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:54,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:09:55,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:09:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:55,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:57,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:09:58,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:58,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:09:59,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:02,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:10,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:10,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:10:10,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:10:14,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:14,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:14,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:17,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:20,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:22,883 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:10:24,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:24,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:10:26,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:10:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:27,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:30,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:33,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:34,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:35,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:38,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:40,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:41,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:10:41,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:10:44,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:45,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:46,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:50,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:50,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:52,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:52,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:10:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:53,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:56,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:10:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:10:56,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:10:57,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:57,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:10:58,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:00,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:00,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:02,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:03,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:03,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:11:04,124 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 23:11:04,332 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 23:11:04,387 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 23:11:04,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 23:11:04,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:06,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:08,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:10,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:12,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:11:12,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:11:12,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:14,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:16,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:18,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:20,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:22,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:23,045 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:11:24,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:26,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:11:27,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:11:28,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:30,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:32,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:34,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:36,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:38,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:40,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:42,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:11:42,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:11:42,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:44,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:46,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:48,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:50,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:52,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:53,122 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:11:54,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:56,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:11:57,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:11:57,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:11:58,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:01,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:03,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:05,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:07,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:09,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:11,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:12:13,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:12:13,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:15,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:17,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:19,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:21,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:23,198 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:12:23,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:25,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:27,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:28,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:12:28,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:12:29,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:31,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:33,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:35,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:37,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:39,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:41,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:43,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:12:43,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:12:43,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:45,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:47,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:49,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:51,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:53,273 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:12:53,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:55,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:57,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:12:58,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:12:58,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:12:59,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:01,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:03,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:05,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:07,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:09,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:12,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:13:14,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:13:14,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:16,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:18,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:20,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:22,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:23,351 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:13:24,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:26,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:28,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:29,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:13:29,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:13:30,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:32,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:34,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:36,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:38,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:40,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:42,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:44,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:13:44,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:13:44,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:46,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:48,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:50,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:52,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:53,418 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:13:54,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:56,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:13:58,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:00,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:14:00,022 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:14:00,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:02,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:04,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:06,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:08,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:10,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:12,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:14,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:15,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:14:15,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:14:16,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:18,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:20,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:22,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:23,492 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:14:24,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:26,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:28,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:14:30,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:14:30,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:32,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:34,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:36,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:38,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:40,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:42,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:44,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:14:45,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:14:46,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:48,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:50,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:52,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:53,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:14:54,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:56,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:14:58,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:15:01,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:15:01,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:03,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:05,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:07,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:09,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:11,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:13,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:15,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:16,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:15:16,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:15:17,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:19,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:21,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:23,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:15:23,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:25,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:27,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:29,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:31,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:15:31,604 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:15:31,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:33,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:35,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:37,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:39,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:41,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:43,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:45,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:46,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:15:46,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:15:47,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:49,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:51,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:53,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:15:53,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:55,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:57,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:15:59,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:01,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:02,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:16:02,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:16:03,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:05,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:07,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:09,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:11,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:13,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:15,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:17,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:16:17,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:16:17,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:19,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:21,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:23,783 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:16:23,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:25,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:27,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:30,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:32,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:16:32,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:16:32,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:34,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:36,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:38,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:40,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:42,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:44,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:46,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:48,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:16:48,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:16:48,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:50,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:52,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:53,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:16:54,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:56,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:16:58,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:00,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:02,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:03,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:17:03,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:17:04,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:06,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:08,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:10,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:12,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:14,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:16,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:18,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:17:18,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:17:18,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:20,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:22,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:23,937 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:17:24,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:26,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:28,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:30,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:32,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:33,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:17:33,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:17:34,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:36,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:38,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:40,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:42,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:44,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:46,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:48,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:17:49,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:17:50,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:52,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:54,014 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:17:54,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:56,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:17:58,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:00,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:02,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:18:04,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:18:04,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:06,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:08,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:10,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:12,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:14,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:16,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:19,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:18:19,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:18:19,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:22,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:24,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:24,090 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:18:26,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:28,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:30,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:32,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:34,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:34,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:18:34,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:18:36,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:38,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:40,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:42,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:44,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:46,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:48,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:50,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:18:50,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:18:52,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:54,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:54,151 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:18:56,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:18:58,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:00,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:02,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:04,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:05,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:19:05,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:19:06,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:08,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:10,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:12,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:14,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:16,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:18,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:20,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:20,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:19:20,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:19:22,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:24,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:24,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:19:26,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:28,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:30,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:32,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:34,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:36,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:36,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:19:36,099 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:19:38,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:40,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:42,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:44,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:46,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:48,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:50,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:19:51,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:19:52,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:54,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:54,304 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:19:56,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:19:58,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:00,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:02,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:04,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:06,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:20:06,589 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:20:07,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:09,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:11,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:13,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:15,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:17,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:19,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:21,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:21,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:20:21,867 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:20:23,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:24,380 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:20:25,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:27,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:29,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:31,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:33,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:35,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:37,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:37,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:20:37,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:20:39,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:41,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:43,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:45,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:47,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:49,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:51,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:52,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:20:52,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:20:53,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:54,456 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:20:55,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:57,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:20:59,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:01,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:03,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:05,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:07,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:07,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:21:07,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:21:09,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:11,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:13,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:15,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:17,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:19,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:21,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:21:22,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:21:23,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:24,527 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:21:25,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:27,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:29,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:31,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:33,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:35,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:37,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:38,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:21:38,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:21:39,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:41,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:43,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:45,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:47,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:49,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:52,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:21:53,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:21:54,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:54,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:21:56,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:21:58,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:00,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:02,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:04,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:06,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:08,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:08,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:22:08,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:22:10,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:12,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:14,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:16,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:18,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:20,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:22,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:24,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:22:24,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:22:24,666 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:22:26,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:28,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:30,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:32,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:34,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:36,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:38,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:39,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:22:39,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:22:40,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:42,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:44,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:46,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:48,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:50,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:52,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:54,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:54,736 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:22:54,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:22:54,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:22:56,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:22:58,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:00,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:02,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:04,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:06,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:08,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:10,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:23:10,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:23:10,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:12,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:14,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:16,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:18,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:20,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:22,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:24,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:24,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:23:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:23:25,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:23:26,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:28,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:30,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:32,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:34,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:36,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:38,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:40,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:23:40,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:23:42,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:44,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:46,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:49,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:51,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:53,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:54,874 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:23:55,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:55,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:23:55,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:23:57,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:23:59,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:01,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:03,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:05,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:07,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:09,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:11,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:11,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:24:11,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:24:13,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:15,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:17,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:19,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:21,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:23,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:24,949 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:24:25,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:26,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:24:26,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:24:27,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:29,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:31,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:33,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:35,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:37,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:39,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:41,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:24:41,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:24:43,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:45,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:47,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:49,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:51,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:53,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:55,018 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:24:55,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:57,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:24:57,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:24:57,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:24:59,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:01,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:03,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:05,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:07,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:09,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:11,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:12,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:25:12,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:25:13,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:15,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:17,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:19,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:21,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:23,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:25,089 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:25:25,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:27,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:27,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:25:27,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:25:29,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:31,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:33,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:35,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:37,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:39,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:41,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:25:42,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:25:44,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:46,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:48,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:50,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:52,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:54,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:55,152 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:25:56,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:58,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:25:58,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:25:58,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:26:00,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:02,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:04,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:06,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:08,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:10,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:12,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:13,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:26:13,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:26:14,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:16,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:18,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:20,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:22,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:24,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:25,228 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:26:26,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:28,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:28,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:26:28,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:26:30,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:32,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:34,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:36,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:38,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:40,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:42,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:44,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:26:44,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:26:44,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:46,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:48,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:50,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:52,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:54,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:55,296 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:26:56,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:58,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:26:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:26:59,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:27:00,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:02,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:04,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:06,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:08,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:10,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:12,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:14,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:14,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:27:14,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:27:16,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:18,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:21,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:23,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:25,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:25,359 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:27:27,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:29,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:27:30,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:27:31,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:33,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:35,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:37,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:39,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:41,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:43,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:45,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:45,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:27:45,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:27:47,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:49,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:51,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:53,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:55,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:55,416 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:27:57,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:27:59,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:00,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:28:00,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:28:01,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:03,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:05,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:07,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:09,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:11,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:13,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:15,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:15,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:28:15,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:28:17,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:19,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:21,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:23,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:25,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:25,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:28:27,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:29,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:31,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:31,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:28:31,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:28:33,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:35,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:37,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:39,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:41,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:43,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:45,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:46,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:28:46,540 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:28:47,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:50,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:52,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:54,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:55,548 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:28:56,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:28:58,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:00,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:01,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:29:01,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:29:02,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:04,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:06,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:08,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:10,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:12,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:14,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:16,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:17,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:29:17,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:29:18,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:20,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:22,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:24,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:25,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:29:26,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:28,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:30,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:32,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:32,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:29:32,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:29:34,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:36,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:38,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:40,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:42,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:44,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:46,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:29:47,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:29:48,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:50,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:52,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:54,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:55,701 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:29:56,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:29:58,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:00,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:02,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:03,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:30:03,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:30:05,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:07,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:09,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:11,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:13,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:15,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:17,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:18,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:30:18,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:30:19,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:21,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:23,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:25,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:25,774 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:30:27,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:29,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:31,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:33,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:33,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:30:33,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:30:35,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:37,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:39,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:41,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:43,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:45,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:47,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:48,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:30:48,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:30:49,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:51,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:53,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:55,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:55,852 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:30:57,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:30:59,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:01,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:03,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:31:04,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:31:05,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:07,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:09,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:11,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:14,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:16,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:18,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:19,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:31:19,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:31:20,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:22,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:24,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:25,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:31:26,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:28,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:30,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:32,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:34,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:31:34,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:31:36,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:38,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:40,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:42,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:44,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:46,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:48,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:31:50,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:31:50,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:52,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:54,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:55,999 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:31:56,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:31:58,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:00,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:02,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:04,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:05,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:32:05,343 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:32:06,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:08,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:10,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:12,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:14,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:16,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:18,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:20,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:32:20,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:32:22,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:24,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:26,067 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:32:26,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:28,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:30,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:32,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:34,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:32:36,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:32:36,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:38,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:40,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:43,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:45,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:47,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:49,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:51,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:32:51,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:32:51,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:53,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:55,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:56,131 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:32:57,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:32:59,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:01,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:03,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:05,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:06,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:33:06,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:33:07,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:09,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:11,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:13,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:15,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:17,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:19,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:21,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:22,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:33:22,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:33:23,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:25,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:26,205 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:33:27,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:29,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:31,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:33,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:35,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:37,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:33:37,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:33:37,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:39,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:41,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:43,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:45,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:47,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:49,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:51,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:33:52,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:33:53,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:55,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:56,282 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:33:57,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:33:59,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:01,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:03,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:05,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:07,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:34:07,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:34:09,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:11,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:13,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:15,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:17,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:19,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:21,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:23,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:34:23,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:34:23,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:25,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:26,359 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:34:27,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:29,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:31,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:33,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:35,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:37,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:34:38,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:34:39,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:41,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:43,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:46,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:48,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:50,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:52,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:53,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:34:53,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:34:54,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:56,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:34:56,436 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:34:58,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:00,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:02,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:04,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:06,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:08,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:09,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:35:09,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:35:10,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:12,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:14,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:16,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:18,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:20,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:22,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:35:24,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:35:24,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:26,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:26,502 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:35:28,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:30,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:32,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:34,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:36,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:38,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:39,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:35:39,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:35:40,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:42,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:44,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:46,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:48,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:50,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:52,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:54,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:54,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:35:54,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:35:56,569 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:35:57,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:35:59,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:01,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:03,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:05,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:07,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:09,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:36:10,082 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:36:11,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:13,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:15,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:17,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:19,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:21,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:23,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:36:25,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:36:25,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:26,643 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:36:27,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:29,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:31,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:33,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:35,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:37,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:39,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:40,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:36:40,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:36:41,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:43,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:45,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:47,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:49,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:51,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:53,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:55,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:55,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:36:55,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:36:56,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:36:57,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:36:59,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:01,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:03,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:05,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:07,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:09,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:11,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:37:11,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:37:11,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:13,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:15,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:17,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:19,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:21,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:23,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:25,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:37:26,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:37:26,875 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:37:27,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:29,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:31,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:33,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:35,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:37,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:40,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:41,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:37:41,968 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:37:42,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:44,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:46,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:48,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:50,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:52,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:54,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:56,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:37:56,866 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:37:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:37:57,212 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:37:58,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:00,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:02,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:04,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:06,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:08,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:10,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:12,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:38:12,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:38:12,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:14,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:16,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:18,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:20,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:22,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:24,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:26,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:26,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:38:27,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:38:27,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:38:28,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:30,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:32,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:34,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:36,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:38,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:40,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:42,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:38:43,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:38:44,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:46,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:48,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:50,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:52,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:54,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:56,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:38:57,017 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:38:58,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:38:58,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:38:58,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:00,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:02,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:05,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:07,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:09,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:11,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:13,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:13,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:39:13,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:39:15,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:17,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:19,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:21,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:23,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:25,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:27,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:39:27,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:28,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:39:28,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:39:29,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:31,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:33,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:35,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:37,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:39,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:41,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:43,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:44,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:39:44,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:39:45,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:47,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:49,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:51,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:53,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:55,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:57,168 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:39:57,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:39:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:39:59,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:39:59,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:01,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:03,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:05,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:07,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:09,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:11,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:14,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:14,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:40:14,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:40:16,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:18,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:20,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:22,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:24,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:26,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:27,244 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:40:28,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:40:30,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:40:30,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:32,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:34,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:36,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:38,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:40,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:42,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:44,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:45,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:40:45,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:40:46,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:48,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:50,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:52,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:54,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:56,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:40:57,318 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:40:58,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:00,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:41:00,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:41:02,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:04,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:06,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:08,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:10,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:12,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:14,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:41:15,945 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:41:16,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:18,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:20,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:22,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:24,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:26,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:27,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:41:28,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:30,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:31,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:41:31,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:41:32,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:34,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:36,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:38,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:40,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:42,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:44,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:46,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:41:46,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:41:46,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:48,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:50,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:52,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:54,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:56,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:41:57,438 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:41:58,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:00,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:01,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:42:01,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:42:02,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:05,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:07,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:09,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:11,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:13,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:15,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:16,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:42:16,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:42:17,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:19,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:21,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:23,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:25,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:27,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:42:27,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:29,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:31,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:32,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:42:32,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:42:33,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:35,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:37,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:39,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:41,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:43,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:45,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:47,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:42:47,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:42:47,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:49,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:51,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:53,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:55,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:57,583 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:42:57,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:42:59,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:01,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:43:02,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:43:03,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:05,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:07,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:09,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:11,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:13,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:15,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:17,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:43:18,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:43:19,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:21,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:23,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:25,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:27,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:43:27,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:29,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:31,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:33,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:43:33,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:43:33,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:35,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:37,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:39,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:41,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:43,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:45,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:47,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:43:48,595 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:43:49,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:51,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:53,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:56,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:43:57,716 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:43:58,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:00,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:02,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:03,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:44:03,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:44:04,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:06,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:08,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:10,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:12,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:14,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:16,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:18,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:19,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:44:19,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:44:20,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:22,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:24,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:26,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:27,778 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:44:28,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:30,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:32,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:34,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:44:34,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:44:34,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:36,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:38,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:40,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:42,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:44,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:46,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:48,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:49,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:44:49,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:44:50,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:52,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:54,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:56,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:44:57,852 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:44:58,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:00,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:02,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:04,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:05,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:45:05,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:45:06,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:08,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:10,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:13,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:15,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:17,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:19,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:45:20,258 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:45:21,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:23,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:25,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:27,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:27,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:45:29,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:31,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:33,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:35,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:45:35,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:45:35,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:37,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:39,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:41,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:43,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:45,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:47,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:49,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:50,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:45:50,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:45:51,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:53,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:55,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:57,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:45:58,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:45:59,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:01,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:03,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:05,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:06,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:46:06,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:46:07,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:09,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:11,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:13,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:15,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:17,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:19,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:21,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:22,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:46:22,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:46:23,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:25,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:27,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:28,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:46:29,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:31,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:33,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:35,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:37,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:46:38,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:46:39,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:41,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:43,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:45,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:47,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:50,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:52,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:54,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:46:54,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:46:54,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:56,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:46:58,150 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:46:58,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:00,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:02,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:04,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:06,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:08,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:09,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:47:09,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:47:10,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:12,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:14,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:16,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:18,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:20,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:22,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:24,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:24,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:47:24,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:47:26,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:28,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:47:28,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:30,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:32,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:34,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:36,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:38,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:47:40,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:47:40,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:42,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:44,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:46,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:48,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:50,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:52,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:54,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:55,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:47:55,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:47:56,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:47:58,299 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:47:58,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:00,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:02,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:04,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:06,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:08,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:10,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:10,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:48:10,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:48:12,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:14,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:17,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:19,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:21,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:23,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:25,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:26,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:48:26,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:48:27,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:28,373 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:48:29,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:31,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:33,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:35,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:37,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:39,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:41,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:42,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:48:42,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:48:43,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:45,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:47,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:49,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:51,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:53,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:55,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:57,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:48:58,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:48:58,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:48:58,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:48:59,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:01,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:03,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:05,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:07,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:09,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:11,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:13,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:15,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:17,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:18,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:49:18,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:49:19,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:21,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:23,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:25,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:27,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:28,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:49:29,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:31,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:33,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:35,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:49:37,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:49:37,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:39,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:42,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:44,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:46,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:48,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:50,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:52,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:54,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:49:54,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:49:54,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:56,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:49:58,584 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:49:58,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:00,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:02,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:04,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:06,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:08,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:09,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:50:09,830 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:50:10,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:12,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:14,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:16,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:18,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:20,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:22,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:24,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:50:25,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:50:26,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:28,657 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:50:28,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:30,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:32,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:34,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:36,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:38,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:40,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:42,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:50:42,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:50:42,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:44,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:46,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:48,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:50,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:52,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:54,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:56,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:50:57,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:50:57,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:50:58,732 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:50:58,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:00,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:02,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:04,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:06,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:08,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:10,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:51:12,757 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:51:12,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:14,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:16,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:18,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:21,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:23,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:25,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:27,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:28,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:51:28,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:51:28,803 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:51:29,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:31,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:33,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:35,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:37,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:39,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:41,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:51:43,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:51:43,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:45,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:47,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:49,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:51,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:53,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:55,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:57,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:51:58,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:51:59,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:51:59,107 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:51:59,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:01,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:03,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:05,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:07,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:09,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:11,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:13,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:14,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:52:14,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:52:15,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:17,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:19,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:21,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:23,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:25,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:27,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:28,949 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:52:29,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:30,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:52:30,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:52:31,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:33,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:35,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:37,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:39,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:41,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:43,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:45,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:52:45,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:52:45,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:47,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:49,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:51,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:53,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:52:59,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:53:00,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:53:00,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:53:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:53:16,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:53:29,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:53:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:53:32,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:53:47,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:53:47,677 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:53:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:53:53,979 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-22 23:53:54,031 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-22 23:53:54,105 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-22 23:53:54,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-22 23:53:55,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:53:59,178 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:53:59,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:54:03,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:54:03,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:54:15,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:54:18,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:54:18,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:54:29,258 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:54:33,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:54:33,731 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:54:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:54:49,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:54:59,335 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:55:04,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:55:04,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:55:20,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:55:20,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:55:29,413 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:55:35,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:55:35,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:55:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:55:50,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:55:59,489 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:56:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:56:06,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:56:21,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:56:21,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:56:29,564 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:56:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:56:37,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:56:52,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:56:52,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:56:59,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:57:08,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:57:08,126 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:57:23,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:57:23,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:57:29,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:57:38,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:57:38,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:57:53,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:57:53,894 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:57:59,790 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:58:09,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:09,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:58:09,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:58:13,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:13,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:15,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:17,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:18,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:18,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:21,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:23,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:24,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:58:24,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:58:25,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:27,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:29,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:29,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:58:31,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:33,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:35,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:37,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:39,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:39,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:58:39,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:58:41,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:41,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:42,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:43,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:44,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:45,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:47,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:49,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:50,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:51,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:51,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:53,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:54,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:58:54,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:58:55,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:58:57,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:59,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:58:59,933 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:59:04,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:06,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:07,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:07,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:10,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:59:10,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:59:12,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:13,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:17,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:18,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:18,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:21,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:22,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:25,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:59:25,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:59:25,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:25,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:29,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:29,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,008 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-22 23:59:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:30,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:31,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:31,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:35,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:35,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:37,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:38,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:39,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:39,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:40,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:59:40,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:59:42,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:46,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:46,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:46,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:48,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:50,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:51,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:51,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:52,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:54,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:56,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-22 23:59:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-22 23:59:56,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-22 23:59:57,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-22 23:59:58,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:00,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:00:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:00,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:01,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:02,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:04,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:06,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:09,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:09,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:10,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:11,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:00:11,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:00:12,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:14,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:14,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:16,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:18,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:18,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:18,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:20,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:22,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:24,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:26,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:00:26,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:00:28,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:30,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:30,149 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:00:32,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:34,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:36,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:37,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:38,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:39,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:39,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:40,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:41,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:00:41,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:00:42,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:43,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:44,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:46,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:48,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:48,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:49,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:50,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:51,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:51,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:53,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:54,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:00:56,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:00:57,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:00:57,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:01:00,219 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:01:02,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:06,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:08,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:10,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:12,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:12,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:01:12,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:01:14,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:16,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:20,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:25,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:26,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:27,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:01:28,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:01:28,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:28,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:30,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:01:31,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:33,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:35,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:37,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:37,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:38,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:41,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:42,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:42,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:43,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:01:43,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:01:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:45,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:47,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:49,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:51,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:51,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:53,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:55,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:56,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:57,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:57,415 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 00:01:57,620 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 00:01:57,693 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 00:01:58,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 00:01:58,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:58,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:01:58,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:01:59,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:01:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:01:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:00,400 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:02:01,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:05,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:05,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:07,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:11,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:11,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:13,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:02:14,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:02:15,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:17,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:19,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:19,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:19,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:21,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:23,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:25,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:27,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:29,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:02:29,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:02:30,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:02:31,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:33,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:34,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:35,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:37,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:39,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:39,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:39,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:40,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:41,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:41,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:42,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:43,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:43,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:44,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:02:44,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:02:45,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:47,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:47,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:49,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:56,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:02:59,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:02:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:02:59,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:03:00,544 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:03:03,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:05,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:07,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:07,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:09,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:11,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:11,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:03:15,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:03:15,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:15,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:19,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:22,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:25,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:26,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:28,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:29,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:03:30,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:03:30,613 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:03:31,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:31,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:33,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:35,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:37,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:37,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:38,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:41,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:41,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:43,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:43,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:44,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:45,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:45,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:03:45,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:03:49,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:51,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:51,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:52,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:53,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:54,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:55,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:57,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:03:57,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:57,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:03:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:00,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:04:01,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:04:01,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:04:01,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:03,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:05,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:05,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:07,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:09,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:09,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:11,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:11,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:11,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:13,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:14,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:14,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:15,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:16,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:04:16,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:04:17,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:19,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:21,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:23,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:25,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:27,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:29,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:30,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:04:31,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:31,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:04:31,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:04:32,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:35,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:36,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:37,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:38,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:39,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:41,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:44,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:04:47,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:04:49,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:49,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:52,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:53,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:56,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:04:58,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:04:58,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:00,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:00,833 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:05:02,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:05:02,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:05:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:04,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:05,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:06,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:08,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:11,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:11,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:12,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:13,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:13,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:16,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:05:17,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:05:18,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:18,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:22,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:22,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:22,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:24,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:26,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:26,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:28,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:29,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:30,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:05:32,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:32,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:32,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:05:32,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:05:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:38,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:40,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:41,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:43,990 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 00:05:44,160 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 00:05:44,216 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 00:05:44,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 00:05:44,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:46,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:48,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:05:48,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:05:48,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:49,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:49,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:50,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:51,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:51,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:52,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:53,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:54,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:56,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:56,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:56,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:58,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:05:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:05:59,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:00,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:00,996 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:06:02,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:02,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:03,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:06:03,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:06:04,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:06,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:07,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:08,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:09,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:10,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:12,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:14,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:16,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:18,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:18,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:06:18,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:06:18,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:18,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:20,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:22,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:24,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:26,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:26,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:26,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:27,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:28,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:30,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:31,061 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:06:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:31,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:32,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:33,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:34,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:06:34,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:06:35,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:37,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:42,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:42,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:45,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:47,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:49,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:06:49,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:06:51,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:51,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:06:55,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:55,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:06:59,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:01,132 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:07:01,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:04,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:07:04,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:07:05,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:06,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:06,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:09,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:11,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:11,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:11,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:12,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:15,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:15,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:16,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:19,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:20,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:07:20,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:07:20,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:21,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:23,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:27,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:28,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:29,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:31,197 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:07:31,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:31,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:31,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:32,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:33,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:35,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:35,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:07:35,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:07:36,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:36,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:37,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:37,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:38,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:39,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:41,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:42,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:45,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:47,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:47,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:49,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:49,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:50,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:07:50,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:07:51,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:53,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:53,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:53,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:54,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:55,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:57,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:07:57,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:57,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:07:59,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:01,270 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:08:01,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:03,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:03,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:03,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:04,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:05,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:08:06,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:08:06,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:06,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:07,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:09,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:11,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:13,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:15,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:15,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:17,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:19,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:21,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:21,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:21,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:08:21,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:08:23,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:25,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:26,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:27,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:28,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:30,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:31,342 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:08:32,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:36,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:08:36,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:08:40,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:43,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:45,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:46,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:48,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:50,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:52,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:08:52,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:08:54,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:55,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:55,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:58,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:08:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:08:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:01,417 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:09:02,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:02,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:04,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:05,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:06,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:07,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:09:07,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:09:08,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:12,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:12,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:14,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:16,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:17,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:18,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:19,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:19,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:20,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:23,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:09:23,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:09:24,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:26,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:28,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:30,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:31,488 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:09:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:32,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:33,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,292 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 00:09:34,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:34,502 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 00:09:34,570 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 00:09:34,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:34,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:35,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 00:09:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:36,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:36,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:09:38,456 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:09:40,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:42,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:43,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:43,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:44,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:45,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:46,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:48,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:48,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:50,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:51,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:52,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:53,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:09:53,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:09:54,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:55,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:55,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:09:56,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:09:58,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:00,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:00,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:00,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:01,584 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:10:02,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:02,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:04,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:06,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:08,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:10:09,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:10:10,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:12,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:14,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:15,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:16,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:18,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:19,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:20,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:22,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:22,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:24,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:10:24,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:10:24,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:26,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:31,654 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:10:32,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:38,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:38,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:39,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:10:39,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:10:43,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:44,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:44,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:47,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:51,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:52,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:10:55,156 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:10:55,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:10:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:10:59,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:01,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:01,730 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:11:03,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:03,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:05,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:06,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:06,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:07,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:09,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:10,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:11:10,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:11:11,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:11,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:12,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:12,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:13,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:15,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:16,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:17,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:18,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:18,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:19,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:21,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:25,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:25,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:11:25,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:11:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:27,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:27,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:29,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:30,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:31,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:31,804 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:11:32,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:32,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:33,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:37,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:39,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:40,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:11:41,127 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:11:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:41,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:42,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:43,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:44,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:44,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:45,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:47,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:49,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:51,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:53,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:54,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:55,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:56,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:56,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:11:56,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:11:57,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:11:57,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:11:59,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:01,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:01,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:12:02,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:03,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:05,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:07,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:09,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:11,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:11,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:12:11,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:12:13,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:13,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:15,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:17,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:17,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:17,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:18,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:20,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:22,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:12:26,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:12:27,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:27,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:30,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:31,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:12:36,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:40,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:41,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:12:42,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:12:42,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:42,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:44,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:48,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:50,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:52,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:52,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:54,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:12:57,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:12:58,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:12:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:12:58,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:00,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:02,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:13:03,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:04,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:04,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:06,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:09,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:09,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:10,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:11,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:11,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:12,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:13:12,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:13:13,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:13,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:16,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:18,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:18,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:20,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:20,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:21,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:22,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:23,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:23,891 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 00:13:24,094 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 00:13:24,168 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 00:13:24,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 00:13:25,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:26,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:28,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:13:28,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:13:28,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:29,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:29,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:30,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,189 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:13:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:32,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:33,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:34,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:34,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:36,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:38,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:39,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:39,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:40,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:42,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:43,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:43,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:13:43,575 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:13:44,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:45,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:45,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:46,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:46,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:48,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:49,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:49,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:50,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:52,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:53,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:53,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:54,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:55,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:56,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:58,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:13:58,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:13:58,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:13:59,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:13:59,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:00,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:01,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:01,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:02,266 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:14:02,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:03,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:04,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:06,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:08,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:10,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:10,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:11,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:13,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:13,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:14,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:14:14,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:14:15,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:25,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:29,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:14:29,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:14:31,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:32,340 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:14:35,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:37,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:37,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:39,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:41,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:43,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:45,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:14:45,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:14:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:46,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:47,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:51,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:51,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:14:53,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:57,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:14:59,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:00,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:15:00,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:15:02,417 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:15:02,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:03,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:05,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:09,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:10,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:10,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:11,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:12,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:15,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:15,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:15:15,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:15:16,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:16,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:17,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:19,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:19,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:19,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:19,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:21,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:22,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:22,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:23,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:25,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:27,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:29,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:29,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:30,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:15:30,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:15:31,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:32,491 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:15:32,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:33,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:35,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:36,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:36,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:37,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:39,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:39,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:40,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:40,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:41,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:43,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:43,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:45,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:15:46,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:15:47,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:47,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:48,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:48,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:49,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:51,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:52,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:52,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:53,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:55,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:56,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:56,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:56,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:56,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:56,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:56,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:57,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:59,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:15:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:15:59,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:01,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:16:01,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:16:02,566 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:16:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:02,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:03,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:05,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:07,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:09,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:10,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:11,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:13,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:17,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:16:17,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:16:23,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:23,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:27,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:31,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:32,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:16:32,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:16:32,753 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:16:35,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:39,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:40,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:43,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:43,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:46,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:47,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:47,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:16:47,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:16:49,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:49,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:50,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:53,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:53,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:16:54,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:16:58,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:00,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:02,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:17:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:17:03,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:17:03,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:03,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:04,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:06,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:06,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:07,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:08,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:12,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:13,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:16,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:16,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 00:17:17,752 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 00:17:17,961 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 00:17:18,032 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 00:17:18,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:17:18,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:17:18,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 00:17:18,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:20,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:22,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:24,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:26,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:28,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:30,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:32,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:32,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:17:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:17:33,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:17:34,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:36,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:38,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:40,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:42,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:44,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:46,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:48,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:49,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:17:49,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:17:50,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:52,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:54,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:56,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:17:58,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:00,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:02,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:02,888 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:18:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:18:04,421 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:18:04,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:06,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:08,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:10,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:12,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:14,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:16,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:18,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:18:19,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:18:20,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:22,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:24,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:26,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:29,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:31,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:32,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:18:33,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:35,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:18:35,028 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:18:35,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:37,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:39,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:41,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:43,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:45,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:47,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:49,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:50,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:18:50,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:18:51,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:53,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:55,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:57,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:18:59,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:01,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:03,035 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:19:03,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:05,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:19:05,639 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:19:05,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:07,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:09,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:11,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:13,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:15,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:17,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:19,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:20,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:19:20,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:19:21,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:23,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:25,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:27,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:29,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:31,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:33,113 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:19:33,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:35,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:36,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:19:36,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:19:37,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:40,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:42,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:44,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:46,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:48,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:50,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:51,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:19:51,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:19:52,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:54,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:56,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:19:58,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:00,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:02,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:03,191 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:20:04,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:06,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:06,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:20:06,683 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:20:08,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:10,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:12,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:14,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:16,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:18,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:20,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:21,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:20:21,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:20:22,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:24,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:26,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:28,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:30,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:32,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:33,269 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:20:34,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:36,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:37,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:20:37,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:20:38,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:40,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:42,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:44,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:46,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:48,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:50,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:52,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:20:52,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:20:52,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:54,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:56,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:20:58,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:00,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:02,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:03,344 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:21:04,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:06,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:07,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:21:07,884 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:21:08,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:10,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:12,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:14,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:16,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:18,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:20,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:22,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:23,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:21:23,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:21:24,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:26,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:28,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:30,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:32,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:33,418 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:21:34,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:36,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:21:38,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:21:39,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:41,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:43,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:45,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:47,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:49,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:51,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:53,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:53,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:21:53,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:21:55,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:57,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:21:59,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:01,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:03,492 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:22:03,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:05,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:07,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:09,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:22:09,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:22:09,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:11,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:13,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:15,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:17,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:19,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:21,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:23,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:24,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:22:24,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:22:25,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:27,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:29,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:31,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:33,567 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:22:33,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:35,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:37,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:39,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:22:39,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:22:39,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:41,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:43,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:45,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:47,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:49,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:51,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:53,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:55,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:22:55,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:22:55,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:57,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:22:59,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:01,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:03,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:23:03,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:05,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:07,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:09,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:10,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:23:10,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:23:11,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:13,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:16,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:18,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:20,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:22,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:24,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:25,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:23:25,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:23:26,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:28,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:30,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:32,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:33,719 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:23:34,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:36,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:38,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:40,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:40,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:23:40,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:23:42,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:44,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:46,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:48,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:50,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:52,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:54,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:56,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:23:56,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:23:56,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:23:58,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:00,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:02,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:03,793 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:24:04,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:06,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:08,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:10,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:11,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:24:11,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:24:12,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:14,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:16,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:18,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:20,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:22,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:24,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:26,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:26,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:24:26,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:24:28,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:30,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:32,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:33,870 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:24:34,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:36,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:38,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:41,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:42,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:24:42,128 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:24:43,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:45,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:47,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:49,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:51,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:53,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:55,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:57,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:24:57,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:24:57,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:24:59,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:01,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:03,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:03,944 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:25:05,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:07,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:09,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:11,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:12,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:25:12,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:25:13,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:15,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:17,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:19,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:21,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:23,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:25,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:27,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:25:27,963 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:25:29,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:31,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:33,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:34,016 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:25:35,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:37,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:39,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:41,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:25:43,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:25:43,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:45,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:47,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:49,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:52,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:54,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:56,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:25:58,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:25:58,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:25:58,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:00,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:02,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:04,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:26:04,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:06,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:08,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:10,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:12,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:13,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:26:13,887 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:26:14,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:16,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:18,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:20,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:22,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:24,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:26,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:28,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:26:29,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:26:30,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:32,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:34,190 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:26:34,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:36,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:38,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:40,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:42,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:44,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:26:44,384 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:26:44,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:46,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:48,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:50,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:52,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:54,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:56,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:58,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:26:59,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:26:59,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:27:00,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:02,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:04,236 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:27:04,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:06,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:08,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:10,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:12,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:14,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:14,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:27:14,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:27:16,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:18,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:20,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:22,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:24,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:26,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:28,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:27:30,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:27:30,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:32,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:34,311 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:27:34,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:36,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:38,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:40,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:42,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:44,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:45,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:27:45,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:27:46,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:48,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:51,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:53,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:55,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:57,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:27:59,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:00,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:28:00,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:28:01,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:03,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:04,387 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:28:05,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:07,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:09,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:11,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:13,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:15,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:16,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:28:16,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:28:17,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:19,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:21,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:23,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:25,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:27,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:29,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:28:31,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:28:31,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:33,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:34,461 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:28:35,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:37,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:39,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:41,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:43,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:45,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:28:46,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:28:47,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:49,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:51,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:53,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:55,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:28:57,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:00,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:02,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:29:02,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:29:02,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:04,522 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:29:04,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:06,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:08,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:10,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:12,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:14,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:16,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:29:17,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:29:18,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:20,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:22,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:24,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:26,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:28,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:30,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:32,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:29:32,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:29:32,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:34,597 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:29:34,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:36,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:38,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:40,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:42,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:44,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:46,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:47,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:29:47,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:29:48,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:50,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:52,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:54,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:56,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:29:58,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:00,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:02,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:30:03,113 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:30:04,666 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:30:04,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:06,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:08,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:10,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:12,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:14,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:16,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:18,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:30:18,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:30:18,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:20,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:22,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:24,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:26,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:28,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:30,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:32,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:33,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:30:33,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:30:34,741 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:30:34,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:36,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:38,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:40,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:43,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:45,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:47,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:49,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:30:49,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:30:49,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:51,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:53,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:55,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:57,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:30:59,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:01,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:03,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:04,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:31:04,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:31:04,996 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:31:05,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:07,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:09,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:11,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:13,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:15,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:17,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:19,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:20,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:31:20,080 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:31:21,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:23,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:25,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:27,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:29,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:31,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:33,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:34,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:31:35,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:31:35,304 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:31:35,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:37,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:39,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:41,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:43,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:45,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:47,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:49,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:50,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:31:50,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:31:51,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:53,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:55,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:57,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:31:59,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:01,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:03,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:04,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:32:05,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:32:05,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:32:05,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:07,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:10,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:12,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:14,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:16,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:18,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:20,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:21,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:32:21,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:32:22,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:24,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:26,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:28,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:30,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:32,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:34,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:35,039 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:32:36,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:32:36,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:32:36,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:38,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:40,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:42,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:44,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:46,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:48,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:50,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:51,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:32:51,580 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:32:52,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:54,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:57,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:32:59,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:01,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:03,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:05,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:05,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:33:06,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:33:06,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:33:07,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:09,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:11,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:13,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:15,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:17,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:19,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:21,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:33:22,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:33:23,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:25,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:27,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:29,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:31,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:33,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:35,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:35,186 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:33:37,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:37,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:33:37,589 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:33:39,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:41,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:43,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:45,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:48,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:50,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:52,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:33:52,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:33:54,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:56,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:33:58,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:00,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:02,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:04,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:05,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:34:06,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:08,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:34:08,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:34:10,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:12,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:14,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:16,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:18,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:20,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:22,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:34:23,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:34:24,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:26,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:28,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:30,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:32,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:34,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:35,333 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:34:36,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:38,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:38,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:34:38,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:34:40,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:42,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:44,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:46,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:48,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:50,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:52,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:54,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:34:54,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:34:54,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:56,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:34:58,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:00,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:02,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:05,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:05,408 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:35:07,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:09,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:09,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:35:09,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:35:11,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:13,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:15,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:17,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:19,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:21,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:23,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:24,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:35:24,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:35:25,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:27,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:29,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:31,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:33,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:35,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:35,483 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:35:37,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:39,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:35:39,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:35:41,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:43,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:45,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:47,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:49,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:51,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:53,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:55,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:55,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:35:55,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:35:57,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:35:59,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:01,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:03,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:05,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:05,560 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:36:07,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:09,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:10,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:36:10,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:36:11,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:13,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:15,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:17,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:19,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:22,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:24,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:25,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:36:25,686 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:36:26,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:28,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:30,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:32,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:34,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:35,632 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:36:36,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:38,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:40,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:41,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:36:41,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:36:42,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:44,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:46,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:48,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:50,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:52,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:54,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:56,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:36:56,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:36:56,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:36:58,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:00,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:02,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:04,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:05,706 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:37:06,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:08,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:10,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:11,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:37:11,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:37:12,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:14,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:16,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:18,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:20,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:22,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:24,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:26,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:26,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:37:26,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:37:28,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:30,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:33,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:35,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:35,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:37:37,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:39,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:41,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:42,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:37:42,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:37:43,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:45,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:47,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:49,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:51,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:53,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:55,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:57,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:37:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:37:57,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:37:59,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:01,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:03,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:05,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:05,851 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:38:07,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:09,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:11,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:38:12,607 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:38:13,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:15,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:17,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:19,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:21,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:23,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:25,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:27,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:27,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:38:27,968 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:38:29,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:31,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:33,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:35,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:35,923 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:38:37,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:39,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:41,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:43,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:43,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:38:43,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:38:45,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:47,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:49,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:51,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:53,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:55,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:58,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:38:58,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:38:58,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:39:00,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:02,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:04,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:05,996 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:39:06,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:08,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:10,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:12,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:13,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:39:13,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:39:14,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:16,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:18,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:20,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:22,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:24,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:26,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:28,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:39:29,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:39:30,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:32,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:34,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:36,070 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:39:36,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:38,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:40,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:42,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:44,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:44,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:39:44,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:39:46,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:48,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:50,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:52,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:54,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:56,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:58,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:39:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:39:59,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:40:00,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:02,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:04,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:06,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:40:06,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:08,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:10,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:12,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:14,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:40:15,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:40:16,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:18,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:20,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:22,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:24,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:26,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:28,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:30,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:30,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:40:30,346 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:40:32,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:34,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:36,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:36,219 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:40:38,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:40,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:42,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:44,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:45,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:40:45,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:40:46,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:48,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:51,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:53,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:55,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:57,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:40:59,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:00,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:41:00,971 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:41:01,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:03,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:05,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:06,295 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:41:07,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:09,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:11,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:13,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:15,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:16,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:41:16,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:41:17,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:19,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:21,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:23,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:25,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:27,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:29,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:31,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:31,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:41:31,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:41:33,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:35,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:36,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:41:37,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:39,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:41,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:43,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:45,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:41:46,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:41:47,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:49,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:51,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:53,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:55,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:41:58,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:00,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:02,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:42:02,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:42:02,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:04,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:06,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:06,440 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:42:08,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:10,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:12,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:14,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:16,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:42:17,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:42:18,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:20,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:22,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:24,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:26,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:28,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:30,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:32,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:32,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:42:32,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:42:34,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:36,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:36,515 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:42:38,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:40,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:42,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:44,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:46,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:48,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:42:48,028 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:42:48,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:50,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:52,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:54,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:56,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:42:58,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:00,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:02,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:03,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:43:03,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:43:04,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:06,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:06,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:43:08,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:10,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:12,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:14,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:16,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:18,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:18,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:43:18,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:43:20,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:23,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:25,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:27,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:29,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:31,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:33,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:33,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:43:33,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:43:35,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:36,668 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:43:37,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:39,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:41,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:43,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:45,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:47,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:43:49,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:43:49,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:51,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:53,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:55,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:57,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:43:59,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:01,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:03,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:04,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:44:04,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:44:05,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:06,741 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:44:07,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:09,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:11,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:13,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:15,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:17,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:19,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:19,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:44:19,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:44:21,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:23,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:25,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:27,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:29,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:31,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:33,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:34,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:44:34,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:44:35,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:36,817 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:44:37,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:40,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:42,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:44,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:46,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:48,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:50,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:44:50,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:44:50,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:52,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:54,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:56,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:44:58,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:00,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:02,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:04,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:45:05,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:45:06,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:06,886 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:45:08,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:10,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:12,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:14,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:16,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:18,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:20,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:45:20,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:45:22,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:24,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:26,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:28,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:30,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:32,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:34,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:36,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:45:36,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:45:36,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:36,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:45:38,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:40,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:42,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:44,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:46,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:48,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:50,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:51,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:45:51,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:45:52,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:54,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:56,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:45:58,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:00,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:02,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:04,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:06,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:46:06,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:46:07,131 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:46:08,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:10,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:12,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:14,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:16,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:18,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:20,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:46:22,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:46:22,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:24,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:27,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:29,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:31,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:33,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:35,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:37,110 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:46:37,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:46:37,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:46:39,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:41,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:43,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:45,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:47,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:49,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:51,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:46:52,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:46:53,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:55,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:57,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:46:59,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:01,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:03,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:05,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:07,187 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:47:07,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:47:08,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:47:09,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:11,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:13,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:15,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:17,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:19,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:21,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:23,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:47:23,304 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:47:23,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:25,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:27,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:29,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:31,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:33,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:35,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:37,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:47:37,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:38,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:47:38,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:47:39,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:41,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:43,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:45,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:47,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:49,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:51,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:53,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:53,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:47:53,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:47:55,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:57,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:47:59,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:01,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:03,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:05,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:07,344 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:48:07,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:09,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:48:09,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:48:09,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:11,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:13,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:15,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:18,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:20,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:22,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:24,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:48:24,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:48:24,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:26,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:28,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:30,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:32,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:34,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:36,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:37,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:48:38,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:48:39,696 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:48:40,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:42,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:44,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:46,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:48,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:50,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:52,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:54,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:48:54,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:48:56,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:48:58,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:00,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:02,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:04,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:06,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:07,473 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:49:08,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:49:10,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:49:10,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:12,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:14,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:16,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:18,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:20,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:22,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:24,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:25,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:49:25,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:49:26,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:28,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:30,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:32,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:34,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:36,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:37,535 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:49:38,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:40,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:40,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:49:40,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:49:42,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:44,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:46,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:48,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:50,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:52,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:54,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:56,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:49:56,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:49:56,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:49:58,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:00,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:02,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:04,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:06,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:07,611 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:50:08,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:10,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:11,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:50:11,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:50:12,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:14,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:17,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:19,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:21,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:23,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:25,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:26,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:50:26,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:50:27,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:29,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:31,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:33,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:35,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:37,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:37,686 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:50:39,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:41,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:41,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:50:41,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:50:43,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:45,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:47,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:49,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:51,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:53,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:55,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:50:57,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:50:57,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:50:59,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:01,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:03,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:05,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:07,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:07,760 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:51:09,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:11,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:12,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:51:12,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:51:13,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:15,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:17,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:19,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:21,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:23,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:25,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:27,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:27,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:51:27,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:51:29,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:31,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:33,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:35,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:37,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:37,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:51:39,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:41,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:42,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:51:42,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:51:43,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:45,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:47,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:49,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:51,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:53,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:55,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:57,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:51:58,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:51:58,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:51:59,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:01,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:03,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:05,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:07,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:07,912 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:52:09,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:11,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:13,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:52:13,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:52:13,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:15,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:17,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:19,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:21,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:23,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:25,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:27,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:52:28,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:52:29,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:32,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:34,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:36,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:37,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:52:38,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:40,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:42,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:44,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:52:44,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:52:44,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:46,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:48,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:50,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:52,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:54,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:56,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:58,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:52:59,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:52:59,307 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:53:00,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:02,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:04,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:06,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:08,063 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:53:08,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:10,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:12,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:14,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:14,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:53:14,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:53:16,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:18,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:20,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:22,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:24,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:26,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:28,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:29,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:53:29,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:53:30,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:32,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:34,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:36,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:38,135 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:53:38,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:40,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:42,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:44,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:53:45,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:53:46,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:48,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:50,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:52,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:54,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:56,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:53:58,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:00,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:54:00,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:54:01,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:03,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:05,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:07,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:08,210 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:54:09,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:11,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:13,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:15,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:15,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:54:15,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:54:17,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:19,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:21,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:23,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:25,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:27,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:29,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:31,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:54:31,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:54:31,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:33,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:35,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:37,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:38,286 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:54:39,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:41,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:43,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:45,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:46,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:54:46,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:54:47,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:49,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:51,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:53,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:55,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:57,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:54:59,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:01,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:55:01,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:55:01,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:03,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:05,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:07,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:08,352 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:55:09,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:11,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:13,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:15,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:16,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:55:16,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:55:18,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:20,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:22,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:24,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:26,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:28,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:30,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:32,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:55:32,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:55:32,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:34,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:36,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:38,424 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:55:38,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:40,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:42,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:44,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:46,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:47,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:55:47,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:55:48,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:50,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:52,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:54,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:56,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:55:58,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:00,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:02,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:56:02,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:56:04,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:06,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:08,488 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:56:08,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:10,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:12,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:14,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:16,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:18,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:56:18,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:56:18,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:20,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:22,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:24,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:26,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:28,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:30,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:32,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:33,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:56:33,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:56:34,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:36,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:38,557 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:56:38,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:40,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:42,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:44,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:47,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:48,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:56:48,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:56:49,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:51,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:53,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:55,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:57,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:56:59,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:01,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:03,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:04,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:57:04,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:57:05,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:07,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:08,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:57:09,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:11,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:13,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:15,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:17,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:57:19,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:57:19,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:21,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:23,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:25,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:27,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:29,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:31,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:33,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:34,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:57:34,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:57:35,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:37,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:38,701 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:57:39,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:41,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:43,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:45,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:48,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:50,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:57:50,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:57:50,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:52,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:54,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:56,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:57:58,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:00,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:02,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:04,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:58:05,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:58:06,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:08,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:08,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:58:10,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:12,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:14,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:16,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:18,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:20,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:20,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:58:20,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:58:22,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:24,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:26,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:28,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:30,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:32,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:34,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:58:36,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:58:36,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:38,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:38,843 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:58:40,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:42,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:44,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:46,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:48,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:50,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:51,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:58:51,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:58:52,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:54,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:56,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:58:58,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:00,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:02,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:04,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:59:06,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:59:06,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:08,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:08,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:59:10,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 00:59:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:59:21,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:59:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:59:37,207 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 00:59:38,979 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 00:59:52,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 00:59:52,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:00:07,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:00:07,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:00:09,055 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:00:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:00:12,124 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 01:00:12,177 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 01:00:12,248 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 01:00:12,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 01:00:13,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:00:15,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:00:19,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:00:23,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:00:23,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:00:35,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:00:38,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:00:38,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:00:39,138 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:00:53,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:00:53,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:01:09,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:01:09,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:01:09,395 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:01:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:01:24,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:01:39,288 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:01:39,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:01:39,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:01:55,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:01:55,409 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:02:09,362 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:02:10,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:02:10,637 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:02:25,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:02:25,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:02:39,441 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:02:41,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:02:41,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:02:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:02:56,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:03:09,519 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:03:11,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:03:11,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:03:26,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:03:26,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:03:39,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:03:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:03:42,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:03:57,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:03:57,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:04:09,678 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:04:12,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:04:12,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:04:26,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:26,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:27,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:27,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:04:27,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:04:29,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:29,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:31,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:33,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:36,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:37,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:39,753 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:04:39,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:41,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:41,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:42,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:42,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:04:43,282 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:04:43,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:43,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:45,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:45,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:45,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:47,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:49,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:49,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:49,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:50,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:51,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:52,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:52,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:53,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:54,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:55,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:56,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:56,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:04:57,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:04:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:04:58,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:04:59,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:01,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:03,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:05,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:05,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:05,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:07,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:09,829 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:05:09,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:10,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:10,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:11,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:12,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:12,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:13,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:13,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:05:13,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:05:15,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:17,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:19,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:27,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:29,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:05:29,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:05:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:33,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:37,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:39,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:05:41,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:44,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:05:44,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:05:45,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:47,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:49,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:53,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:55,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:05:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:56,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:05:59,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:05:59,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:05:59,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:01,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:02,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:02,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:03,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:05,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:07,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:09,971 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:06:11,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:13,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:15,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:06:15,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:06:15,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:16,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:17,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:20,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:22,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:22,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:23,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:24,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:25,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:26,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:27,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:06:30,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:06:30,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:32,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:33,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:34,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:36,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:37,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:38,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:39,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:39,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,037 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:06:40,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:40,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:42,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:42,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:43,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:43,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:44,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:45,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:06:45,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:06:46,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:47,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:48,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:49,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:49,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:50,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:51,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:51,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:52,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:54,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:54,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:55,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:55,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:56,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:57,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:57,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:58,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:06:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:06:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:00,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:00,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:07:00,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:07:02,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:03,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:04,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:06,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:08,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:08,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,107 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:07:10,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:10,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:12,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:14,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:16,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:07:16,207 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:07:16,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:26,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:30,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:07:31,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:07:36,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:40,178 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:07:40,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:42,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:43,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:46,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:07:46,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:07:46,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:48,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:50,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:54,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:56,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:07:59,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:07:59,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:00,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:01,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:08:01,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:08:02,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:02,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:04,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:06,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:08,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:10,252 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:08:10,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:14,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:16,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:16,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:08:17,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:08:17,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:17,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:19,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:19,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:19,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:20,395 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 01:08:20,601 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 01:08:20,673 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 01:08:20,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 01:08:21,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:21,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:22,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:22,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:23,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:23,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:24,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:25,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:26,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:27,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:28,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:29,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:30,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:31,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:31,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:08:32,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:08:32,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:32,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:33,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:34,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:34,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:35,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:36,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:36,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:37,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:39,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:40,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,332 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:08:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:40,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:41,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:43,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:43,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:44,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:44,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:45,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:47,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:08:47,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:08:47,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:48,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:48,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:49,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:50,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:50,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:51,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:52,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:53,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:54,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:54,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:55,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:57,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:57,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:08:58,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:08:59,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:01,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:01,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:03,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:09:03,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:09:03,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:05,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:07,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:09,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:10,401 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:09:11,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:13,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:09:18,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:09:23,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:27,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:31,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:33,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:09:33,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:09:35,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:39,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:40,474 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:09:43,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:47,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:49,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:09:49,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:09:51,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:09:51,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:53,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:57,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:09:59,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:02,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:03,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:04,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:10:04,606 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:10:05,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:06,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:07,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:10,551 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:10:11,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:13,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:15,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:16,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:17,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:18,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:18,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:19,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:10:19,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:10:21,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:22,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:22,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:23,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:25,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:25,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:27,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:27,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:29,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:31,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:32,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:33,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:35,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:10:35,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:10:35,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:35,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:36,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:38,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:40,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:10:40,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:41,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:41,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:42,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:44,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:46,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:47,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:48,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:49,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:49,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:50,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:10:50,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:10:50,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:51,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:51,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:52,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:54,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:55,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:56,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:10:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:57,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:10:58,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:00,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:02,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:05,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:11:05,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:11:05,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:05,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:06,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:07,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:09,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:10,694 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:11:15,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:15,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:19,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:20,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:11:20,944 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:11:23,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:27,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:31,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:35,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:36,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:11:36,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:11:39,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:40,770 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:11:43,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:45,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:49,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:49,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:50,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:51,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:51,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:11:51,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:11:55,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:57,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:11:59,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:11:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:01,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:03,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:04,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:06,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:12:06,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:12:07,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:09,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:10,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:12:11,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:11,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:11,812 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 01:12:12,023 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 01:12:12,087 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 01:12:13,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 01:12:13,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:15,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:15,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:19,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:20,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:20,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:21,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:22,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:12:22,077 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:12:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:23,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:23,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:25,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:25,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:26,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:27,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:28,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:28,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:29,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:30,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:31,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:32,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:33,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:34,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:34,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:35,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:35,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:37,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:12:37,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:12:38,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:38,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:39,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:40,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:12:41,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:41,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:41,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:42,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:43,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:44,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:45,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:46,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:47,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:49,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:49,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:51,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:12:52,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:12:52,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:53,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:53,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:55,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:12:57,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:12:59,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:01,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:03,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:04,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:05,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:07,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:13:07,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:13:11,034 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:13:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:12,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:15,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:15,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:16,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:20,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:20,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:23,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:13:23,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:13:24,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:28,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:32,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:34,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:38,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:13:38,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:13:40,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:41,109 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:13:44,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:44,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:44,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:46,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:48,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:48,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:50,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:52,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:53,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:13:53,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:13:54,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:13:56,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:13:58,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:00,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:01,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:02,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:02,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:04,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:06,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:08,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:14:08,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:14:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:10,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:11,182 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:14:12,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:13,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:14,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:16,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:16,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:17,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:18,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:19,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:20,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:21,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:22,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:24,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:24,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:14:24,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:14:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:25,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:26,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:27,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:28,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:30,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:31,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:32,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:33,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:35,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:36,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:37,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:38,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:14:39,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:14:39,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:40,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:41,251 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:14:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:41,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:42,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:43,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:44,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:45,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:46,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:47,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:47,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:48,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:49,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:50,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:52,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:52,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:52,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:14:54,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:14:54,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:14:54,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:14:56,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:00,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:00,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:05,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:06,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:06,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:09,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:15:10,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:15:11,323 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:15:13,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:15,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:17,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:19,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:19,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:21,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:25,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:25,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:15:25,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:15:29,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:31,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:35,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:39,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:40,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:15:40,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:15:41,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:41,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:15:43,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:43,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:47,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:49,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:52,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:53,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:55,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:55,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:15:56,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:15:56,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:56,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:57,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:15:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:57,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:15:58,411 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 01:15:58,617 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 01:15:58,685 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 01:15:59,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 01:15:59,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:00,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:00,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:01,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:03,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:05,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:07,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:09,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:11,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:11,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:16:11,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:16:11,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:16:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:12,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:13,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:15,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:17,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:18,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:18,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:19,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:21,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:21,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:22,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:23,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:25,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:16:26,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:16:27,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:27,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:29,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:30,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:30,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:31,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:32,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:32,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:33,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:34,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:34,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:35,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:36,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:37,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:37,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:38,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:39,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:41,591 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:16:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:16:41,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:16:43,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:44,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:45,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:45,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:46,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:47,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:47,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:49,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:51,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:16:57,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:16:57,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:16:58,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:16:58,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:00,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:03,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:06,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:07,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:07,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:10,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:11,668 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:17:12,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:17:12,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:17:14,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:15,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:15,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:18,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:22,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:26,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:27,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:17:27,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:17:28,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:31,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:31,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:32,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:34,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:38,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:40,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:41,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:17:43,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:17:43,100 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:17:44,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:46,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:48,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:49,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:52,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:53,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:53,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:53,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:54,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:56,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:58,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:17:58,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:17:58,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:17:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:17:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:00,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:02,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:02,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:04,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:08,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:10,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:11,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:11,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:18:12,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:18:13,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:18:14,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:14,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:14,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:15,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:16,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:18,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:18,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:20,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:20,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:20,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:22,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:23,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:24,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:25,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:25,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:26,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:27,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:28,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:29,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:18:29,020 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:18:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:29,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:30,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:32,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:32,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:36,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:38,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:38,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:38,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:40,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:40,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:18:41,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:42,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:44,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:18:44,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:18:44,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:46,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:53,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:18:56,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:18:59,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:18:59,479 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:19:01,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:07,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:07,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:11,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:11,976 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:19:13,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:13,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:14,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:19:14,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:19:15,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:17,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:21,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:24,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:25,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:27,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:29,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:19:29,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:19:31,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:33,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:35,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:39,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:41,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:42,045 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:19:43,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:45,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:19:45,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:19:45,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:46,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:47,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:49,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:49,781 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 01:19:49,985 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 01:19:50,046 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 01:19:50,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 01:19:51,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:53,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:55,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:57,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:59,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:19:59,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:19:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:00,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:20:00,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:20:01,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:01,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:03,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:05,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:05,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:05,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:07,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:09,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:10,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:10,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:11,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:12,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:12,129 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:20:13,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:13,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:14,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:14,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:20:15,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:20:16,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:17,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:17,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:19,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:20,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:21,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:21,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:22,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:24,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:25,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:26,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:26,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:28,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:30,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:30,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:30,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:20:31,066 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:20:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:32,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:32,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:34,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:34,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:36,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:36,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:37,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:38,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:38,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:40,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:42,205 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:20:46,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:20:46,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:20:50,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:51,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:20:56,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:20:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:00,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:01,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:21:01,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:21:04,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:12,282 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:21:12,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:13,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:13,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:16,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:16,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:21:16,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:21:18,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:21,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:21,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:22,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:24,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:27,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:27,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:28,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:30,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:32,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:21:32,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:21:34,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:36,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:36,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:40,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:42,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:42,354 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:21:43,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:43,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:44,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:44,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:44,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:44,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:46,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:47,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:21:47,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:21:48,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:52,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:54,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:56,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:58,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:21:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:21:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:00,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:01,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:01,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:22:02,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:22:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:05,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:07,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:09,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:11,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:11,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:11,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:12,429 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:22:13,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:13,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:15,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:17,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:17,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:22:17,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:22:19,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:19,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:21,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:21,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:22,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:23,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:24,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:24,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:25,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:29,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:31,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:31,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:33,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:22:33,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:22:33,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:34,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:35,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:36,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:36,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:37,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:42,505 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:22:45,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:47,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:47,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:48,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:22:48,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:22:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:49,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:53,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:53,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:57,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:22:58,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:22:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:01,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:03,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:23:03,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:23:05,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:07,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:10,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:10,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:11,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:12,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:12,575 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:23:15,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:16,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:16,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:17,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:18,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:23:18,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:23:21,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:23,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:25,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:29,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:31,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:31,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:33,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:23:34,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:23:35,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 01:23:36,943 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 01:23:37,162 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 01:23:37,227 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 01:23:37,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 01:23:37,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:38,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:39,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:41,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:42,742 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:23:43,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:45,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:47,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:49,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:49,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:23:49,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:23:51,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:53,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:55,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:57,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:23:59,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:01,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:04,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:24:04,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:24:06,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:08,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:10,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:12,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:12,818 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:24:14,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:16,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:18,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:20,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:24:20,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:24:20,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:22,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:24,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:26,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:28,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:30,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:32,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:34,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:35,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:24:35,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:24:36,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:38,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:40,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:42,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:42,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:24:44,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:46,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:48,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:50,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:50,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:24:50,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:24:52,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:54,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:56,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:24:58,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:00,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:02,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:04,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:05,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:25:05,996 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:25:06,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:08,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:10,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:12,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:12,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:25:14,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:16,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:18,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:20,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:25:21,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:25:22,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:24,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:26,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:28,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:30,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:32,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:34,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:36,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:25:36,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:25:38,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:40,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:42,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:43,031 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:25:44,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:46,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:48,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:51,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:51,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:25:51,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:25:53,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:55,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:57,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:25:59,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:01,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:03,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:05,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:07,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:26:07,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:26:07,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:09,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:11,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:13,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:26:13,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:15,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:17,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:19,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:21,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:23,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:25,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:26,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:26:26,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:26:27,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:29,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:31,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:33,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:35,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:37,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:39,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:41,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:42,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:26:42,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:26:43,186 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:26:43,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:45,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:47,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:49,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:51,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:53,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:55,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:57,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:26:57,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:26:57,567 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:26:59,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:01,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:03,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:05,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:07,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:09,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:11,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:12,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:27:12,805 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:27:13,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:27:13,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:15,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:17,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:19,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:21,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:23,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:26,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:27:28,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:27:28,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:30,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:32,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:34,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:36,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:38,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:40,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:42,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:43,336 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:27:43,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:27:43,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:27:44,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:46,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:48,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:50,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:52,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:54,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:56,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:58,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:27:58,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:27:58,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:28:00,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:02,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:04,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:06,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:08,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:10,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:12,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:13,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:28:13,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:28:13,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:28:14,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:16,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:18,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:20,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:22,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:24,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:26,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:28,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:28:29,345 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:28:30,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:32,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:34,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:36,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:38,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:40,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:42,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:43,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:28:44,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:44,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:28:44,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:28:46,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:48,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:51,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:53,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:55,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:57,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:28:59,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:00,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:29:00,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:29:01,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:03,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:05,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:07,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:09,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:11,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:13,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:13,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:29:15,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:29:15,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:29:15,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:17,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:19,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:21,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:23,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:25,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:27,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:29,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:30,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:29:30,607 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:29:31,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:33,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:35,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:37,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:39,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:41,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:43,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:43,635 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:29:45,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:45,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:29:45,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:29:47,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:49,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:51,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:53,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:55,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:57,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:29:59,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:01,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:30:01,096 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:30:01,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:03,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:05,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:07,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:09,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:11,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:13,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:13,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:30:15,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:16,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:30:16,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:30:17,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:19,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:21,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:23,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:25,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:27,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:29,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:31,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:30:31,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:30:33,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:35,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:37,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:39,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:41,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:43,791 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:30:44,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:46,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:47,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:30:47,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:30:48,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:50,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:52,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:54,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:56,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:30:58,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:00,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:02,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:31:02,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:31:04,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:06,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:08,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:10,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:12,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:13,867 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:31:14,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:16,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:17,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:31:17,852 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:31:18,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:20,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:22,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:24,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:26,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:28,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:30,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:32,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:33,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:31:33,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:31:34,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:36,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:38,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:40,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:42,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:43,944 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:31:44,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:46,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:48,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:31:48,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:31:48,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:50,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:52,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:54,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:56,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:31:58,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:00,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:02,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:03,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:32:03,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:32:04,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:06,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:08,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:10,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:12,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:14,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:32:14,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:17,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:18,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:32:18,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:32:19,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:21,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:23,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:25,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:27,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:29,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:31,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:33,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:32:34,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:32:35,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:37,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:39,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:41,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:43,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:44,096 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:32:45,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:47,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:49,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:49,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:32:49,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:32:51,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:53,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:55,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:57,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:32:59,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:01,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:03,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:04,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:33:04,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:33:05,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:07,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:09,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:11,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:13,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:14,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:33:15,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:17,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:19,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:20,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:33:20,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:33:21,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:23,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:25,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:27,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:29,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:31,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:33,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:35,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:35,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:33:35,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:33:37,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:39,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:41,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:43,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:44,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:33:45,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:47,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:49,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:50,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:33:50,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:33:51,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:53,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:55,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:57,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:33:59,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:01,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:03,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:05,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:06,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:34:06,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:34:07,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:09,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:11,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:14,327 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:34:14,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:16,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:18,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:20,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:21,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:34:21,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:34:22,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:24,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:26,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:28,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:30,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:32,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:34,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:36,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:37,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:34:37,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:34:38,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:40,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:42,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:44,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:34:44,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:46,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:48,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:50,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:52,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:34:52,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:34:52,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:54,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:56,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:34:58,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:00,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:02,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:04,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:06,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:07,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:35:07,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:35:08,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:10,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:12,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:14,478 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:35:14,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:16,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:18,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:20,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:23,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:35:23,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:35:23,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:25,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:27,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:29,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:31,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:33,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:35,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:37,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:38,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:35:38,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:35:39,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:41,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:43,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:44,554 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:35:45,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:47,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:49,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:51,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:53,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:35:53,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:35:53,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:55,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:57,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:35:59,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:01,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:03,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:05,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:07,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:08,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:36:08,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:36:09,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:11,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:13,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:14,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:36:15,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:17,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:19,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:21,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:23,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:24,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:36:24,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:36:25,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:27,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:29,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:31,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:33,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:35,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:37,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:36:39,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:36:39,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:41,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:43,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:44,696 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:36:45,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:47,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:49,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:51,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:53,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:54,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:36:54,794 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:36:55,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:57,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:36:59,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:01,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:03,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:05,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:07,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:09,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:37:10,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:37:11,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:14,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:14,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:37:16,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:18,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:20,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:22,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:24,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:25,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:37:25,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:37:26,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:28,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:30,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:32,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:34,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:36,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:38,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:40,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:37:40,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:37:40,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:42,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:44,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:44,848 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:37:46,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:48,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:50,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:52,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:54,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:37:56,083 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:37:56,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:37:58,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:00,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:02,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:04,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:06,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:08,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:10,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:11,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:38:11,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:38:12,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:14,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:14,915 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:38:16,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:18,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:20,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:22,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:24,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:38:26,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:38:26,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:28,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:30,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:32,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:34,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:36,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:38,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:40,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:41,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:38:41,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:38:42,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:44,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:44,990 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:38:46,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:48,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:50,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:52,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:54,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:56,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:38:57,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:38:57,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:38:58,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:00,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:02,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:04,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:06,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:09,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:11,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:39:12,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:39:13,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:15,067 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:39:15,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:17,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:19,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:21,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:23,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:25,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:27,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:39:27,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:39:27,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:29,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:31,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:33,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:35,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:37,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:39,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:41,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:39:42,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:39:43,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:45,142 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:39:45,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:47,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:49,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:51,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:53,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:55,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:57,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:39:58,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:39:58,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:39:59,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:01,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:03,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:05,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:07,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:09,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:11,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:13,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:40:13,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:40:13,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:15,217 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:40:15,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:17,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:19,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:21,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:23,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:25,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:27,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:28,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:40:28,639 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:40:29,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:31,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:33,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:35,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:37,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:39,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:41,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:43,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:43,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:40:43,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:40:45,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:40:45,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:47,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:49,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:51,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:53,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:55,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:57,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:40:59,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:40:59,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:40:59,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:01,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:03,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:05,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:07,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:10,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:12,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:14,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:41:14,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:41:14,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:15,363 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:41:16,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:18,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:20,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:22,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:24,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:26,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:28,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:29,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:41:29,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:41:30,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:32,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:34,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:36,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:38,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:40,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:42,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:44,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:45,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:41:45,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:41:45,441 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:41:46,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:48,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:50,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:52,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:54,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:56,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:41:58,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:42:00,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:42:00,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:02,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:04,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:06,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:08,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:10,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:12,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:14,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:15,517 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:42:15,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:42:15,540 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:42:16,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:18,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:20,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:22,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:24,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:26,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:29,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:30,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:42:30,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:42:31,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:33,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:35,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:37,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:39,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:41,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:43,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:45,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:42:45,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:42:46,100 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:42:47,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:49,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:51,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:53,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:55,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:57,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:42:59,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:01,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:43:01,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:43:01,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:03,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:05,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:07,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:09,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:11,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:13,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:15,666 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:43:15,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:16,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:43:16,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:43:17,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:19,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:21,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:23,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:25,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:27,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:29,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:31,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:31,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:43:31,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:43:33,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:35,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:37,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:39,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:41,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:43,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:45,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:43:45,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:47,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:43:47,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:43:47,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:49,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:51,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:53,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:56,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:43:58,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:00,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:02,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:44:02,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:44:02,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:04,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:06,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:08,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:10,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:12,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:14,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:15,819 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:44:16,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:17,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:44:17,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:44:18,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:20,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:22,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:24,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:26,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:28,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:30,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:32,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:33,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:44:33,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:44:34,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:36,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:38,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:40,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:42,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:44,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:45,896 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:44:46,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:48,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:44:48,345 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:44:48,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:50,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:52,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:54,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:56,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:44:58,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:00,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:02,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:03,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:45:03,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:45:04,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:06,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:08,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:10,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:12,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:15,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:15,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:45:17,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:45:19,063 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:45:19,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:21,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:23,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:25,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:27,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:29,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:31,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:33,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:34,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:45:34,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:45:35,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:37,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:39,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:41,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:43,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:45,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:46,048 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:45:47,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:49,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:45:49,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:45:49,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:51,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:53,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:55,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:57,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:45:59,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:01,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:03,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:46:04,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:46:05,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:07,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:09,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:11,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:13,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:15,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:16,122 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:46:17,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:19,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:20,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:46:20,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:46:21,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:23,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:25,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:27,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:29,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:31,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:33,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:35,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:46:35,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:46:35,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:37,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:39,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:41,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:43,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:45,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:46,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:46:47,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:49,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:50,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:46:50,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:46:51,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:54,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:56,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:46:58,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:00,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:02,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:04,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:47:05,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:47:06,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:08,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:10,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:12,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:14,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:16,258 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:47:16,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:18,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:20,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:21,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:47:21,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:47:22,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:24,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:26,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:28,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:30,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:32,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:34,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:36,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:47:36,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:47:36,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:38,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:40,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:42,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:45,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:46,332 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:47:47,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:49,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:51,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:47:51,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:47:51,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:53,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:55,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:57,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:47:59,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:01,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:03,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:05,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:48:07,154 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:48:07,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:09,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:11,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:13,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:15,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:16,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:48:17,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:19,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:21,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:22,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:48:22,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:48:23,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:25,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:28,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:30,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:32,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:34,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:36,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:37,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:48:37,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:48:38,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:40,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:42,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:44,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:46,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:46,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:48:48,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:50,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:52,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:53,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:48:53,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:48:54,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:56,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:48:58,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:00,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:02,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:04,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:06,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:08,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:08,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:49:08,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:49:10,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:12,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:14,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:16,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:16,557 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:49:18,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:20,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:22,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:23,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:49:23,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:49:24,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:26,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:29,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:31,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:33,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:35,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:37,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:39,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:39,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:49:39,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:49:41,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:43,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:45,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:46,632 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:49:47,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:49,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:51,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:53,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:54,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:49:54,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:49:55,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:57,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:49:59,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:01,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:03,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:05,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:07,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:09,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:50:09,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:50:11,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:13,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:15,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:16,707 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:50:17,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:19,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:21,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:23,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:50:25,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:50:25,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:27,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:29,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:31,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:33,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:35,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:37,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:39,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:50:40,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:50:41,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:43,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:45,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:46,781 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:50:47,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:49,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:51,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:53,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:55,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:50:55,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:50:57,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:50:59,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:01,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:03,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:05,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:08,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:10,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:10,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:51:10,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:51:12,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:14,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:16,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:16,856 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:51:18,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:20,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:22,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:24,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:26,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:51:26,002 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:51:26,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:28,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:30,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:32,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:34,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:36,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:38,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:40,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:41,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:51:41,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:51:42,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:44,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:46,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:46,936 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:51:48,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:50,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:52,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:54,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:56,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:51:56,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:51:56,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:51:58,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:00,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:02,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:04,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:06,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:08,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:10,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:11,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:52:11,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:52:12,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:14,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:16,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:17,011 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:52:18,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:20,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:22,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:24,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:26,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:27,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:52:27,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:52:28,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:30,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:33,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:35,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:37,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:39,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:41,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:52:42,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:52:43,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:45,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:47,079 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:52:47,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:49,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:51,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:53,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:55,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:57,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:52:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:52:57,856 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:52:59,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:01,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:03,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:05,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:07,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:09,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:11,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:13,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:53:13,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:53:13,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:15,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:17,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:17,155 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:53:19,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:21,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:23,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:25,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:27,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:28,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:53:28,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:53:29,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:31,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:33,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:35,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:37,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:39,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:41,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:43,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:43,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:53:43,692 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:53:45,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:47,230 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:53:48,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:50,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:52,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:54,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:56,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:58,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:53:58,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:53:58,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:54:00,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:02,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:04,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:06,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:08,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:10,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:12,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:14,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:14,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:54:14,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:54:16,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:17,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:54:18,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:20,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:22,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:24,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:26,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:28,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:29,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:54:29,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:54:30,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:32,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:34,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:36,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:38,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:40,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:42,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:44,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:45,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:54:45,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:54:46,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:47,370 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:54:48,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:50,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:52,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:54,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:56,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:54:58,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:00,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:00,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:55:00,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:55:02,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:04,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:06,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:09,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:11,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:13,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:15,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:15,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:55:15,681 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:55:17,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:17,445 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:55:19,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:21,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:23,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:25,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:27,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:29,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:30,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:55:30,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:55:31,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:33,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:35,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:37,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:39,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:41,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:43,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:45,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:46,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:55:46,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:55:47,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:47,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:55:49,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:51,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:53,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:55,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:57,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:55:59,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:01,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:01,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:56:01,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:56:03,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:05,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:07,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:09,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:11,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:13,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:15,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:16,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:56:16,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:56:17,594 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:56:18,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:20,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:22,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:24,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:26,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:28,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:30,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:31,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:56:31,944 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:56:32,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:34,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:36,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:38,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:40,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:42,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:44,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:46,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:47,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:56:47,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:56:47,670 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:56:48,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:50,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:52,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:54,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:56,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:56:58,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:00,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:02,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:57:02,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:57:04,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:06,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:08,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:10,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:12,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:14,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:16,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:17,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:57:17,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:57:17,971 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:57:18,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:20,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:22,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:24,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:26,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:28,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:30,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:32,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:32,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:57:32,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:57:34,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:36,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:38,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:40,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:42,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:44,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:46,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:47,823 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:57:48,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:48,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:57:48,311 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:57:51,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:53,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:55,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:57,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:57:59,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:01,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:03,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:03,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:58:03,575 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:58:05,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:07,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:09,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:11,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:13,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:15,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:17,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:17,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:58:18,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:58:18,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:58:19,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:21,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:23,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:25,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:27,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:29,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:31,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:33,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:34,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:58:34,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:58:35,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:37,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:39,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:41,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:43,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:45,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:47,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:47,962 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:58:49,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:49,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:58:49,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:58:51,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:53,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:55,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:57,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:58:59,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:01,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:03,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:59:04,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:59:05,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:07,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:09,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:11,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:13,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:15,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:17,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:18,037 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:59:19,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:20,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:59:20,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:59:21,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:23,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:25,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:27,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:29,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:31,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:33,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:35,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:59:35,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:59:35,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:37,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:39,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:41,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:43,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:45,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:47,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:48,115 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 01:59:49,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:50,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 01:59:50,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 01:59:51,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:53,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:56,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 01:59:58,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:00,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:02,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:04,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:05,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:00:05,786 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:00:06,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:08,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:10,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:12,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:14,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:16,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:18,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:00:18,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:20,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:00:21,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:00:22,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:24,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:26,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:28,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:30,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:32,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:34,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:36,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:36,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:00:36,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:00:38,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:40,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:42,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:44,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:46,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:48,265 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:00:48,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:50,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:51,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:00:51,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:00:52,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:54,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:56,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:00:58,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:00,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:02,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:04,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:06,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:07,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:01:07,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:01:08,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:10,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:12,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:14,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:16,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:18,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:18,340 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:01:20,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:22,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:01:22,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:01:24,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:26,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:28,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:30,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:32,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:34,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:36,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:37,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:01:37,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:01:38,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:40,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:42,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:44,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:46,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:48,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:48,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:01:50,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:52,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:53,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:01:53,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:01:54,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:56,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:01:58,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:00,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:02,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:04,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:07,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:08,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:02:08,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:02:09,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:11,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:13,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:15,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:17,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:18,489 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:02:19,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:21,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:23,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:23,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:02:23,890 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:02:25,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:27,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:29,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:31,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:33,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:35,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:37,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:39,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:39,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:02:39,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:02:41,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:43,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:45,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:47,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:48,569 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:02:49,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:51,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:53,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:54,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:02:54,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:02:55,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:57,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:02:59,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:01,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:03,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:05,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:07,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:09,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:03:10,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:03:11,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:13,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:15,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:17,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:18,643 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:03:19,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:22,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:24,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:25,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:03:25,523 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:03:26,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:28,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:30,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:32,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:34,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:36,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:38,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:40,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:40,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:03:40,757 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:03:42,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:44,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:46,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:48,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:48,720 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:03:50,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:52,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:54,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:03:56,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:03:56,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:03:58,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:00,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:02,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:04,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:06,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:08,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:10,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:11,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:04:11,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:04:12,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:14,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:16,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:18,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:18,796 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:04:20,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:22,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:24,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:26,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:27,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:04:27,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:04:28,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:31,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:33,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:35,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:37,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:39,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:41,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:42,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:04:42,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:04:43,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:45,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:47,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:48,871 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:04:49,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:51,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:53,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:55,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:57,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:04:57,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:04:57,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:04:59,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:01,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:03,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:05,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:07,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:09,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:11,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:12,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:05:12,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:05:13,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:15,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:17,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:18,948 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:05:19,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:21,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:23,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:25,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:27,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:28,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:05:28,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:05:29,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:31,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:05:43,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:05:43,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:05:49,024 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:05:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:05:58,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:06:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:06:14,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:06:19,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:06:29,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:06:29,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:06:30,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:06:30,858 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 02:06:30,910 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 02:06:30,984 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 02:06:31,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 02:06:32,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:06:33,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:06:38,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:06:44,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:06:44,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:06:49,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:06:54,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:07:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:07:00,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:07:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:07:15,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:07:19,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:07:30,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:07:30,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:07:45,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:07:45,816 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:07:49,343 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:08:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:08:01,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:08:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:08:16,261 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:08:19,421 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:08:31,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:08:31,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:08:46,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:08:46,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:08:49,499 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:09:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:09:01,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:09:17,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:09:17,208 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:09:19,577 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:09:32,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:09:32,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:09:47,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:09:47,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:09:49,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:10:02,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:10:02,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:10:18,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:10:18,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:10:19,733 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:10:33,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:10:33,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:10:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:10:48,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:10:49,810 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:10:50,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:10:52,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:10:54,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:10:56,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:10:58,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:58,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:10:59,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:10:59,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:00,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:02,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:03,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:11:03,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:11:04,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:05,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:06,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:07,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:07,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:08,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:10,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:10,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:12,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:14,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:15,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:16,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:18,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:18,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:19,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:11:19,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:11:19,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:11:20,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:22,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:24,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:24,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:24,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:25,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:26,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:26,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:28,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:28,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:29,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:30,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:31,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:31,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:32,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:34,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:11:34,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:11:34,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:34,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:35,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:37,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:38,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:41,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:41,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:46,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:49,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:11:49,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:11:50,105 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:11:50,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:54,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:11:54,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:54,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:11:58,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:02,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:12:05,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:12:06,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:07,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:08,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:12,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:16,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:18,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:18,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:18,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:20,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:12:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:12:20,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:12:22,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:23,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:24,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:26,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:30,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:32,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:35,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:35,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:12:35,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:12:37,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:37,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:37,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:37,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:37,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:41,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:43,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:45,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:47,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:49,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:50,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:12:51,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:12:51,208 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:12:51,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:53,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:54,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:54,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:55,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:57,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:12:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:58,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:12:59,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:01,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:03,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:04,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:05,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:06,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:13:06,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:13:07,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:07,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:09,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:10,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:10,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:11,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:12,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:13,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:15,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:17,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:17,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:17,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,165 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:13:20,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:20,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:21,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:13:21,687 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:13:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:23,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:23,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:25,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:27,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:27,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:28,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:29,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:31,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:33,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:33,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:36,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:13:36,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:13:39,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:41,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:41,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:43,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:46,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:46,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:47,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:48,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:48,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:50,238 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:13:51,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:52,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:13:52,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:13:55,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:55,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:13:57,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:57,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:13:59,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:03,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:03,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:05,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:07,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:14:07,557 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:14:08,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:08,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:09,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:11,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:14,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:15,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:16,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:17,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:19,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:19,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:20,309 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:14:22,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:14:22,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:14:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:23,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:25,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:27,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:29,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:30,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:30,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:31,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:31,088 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 02:14:31,283 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 02:14:31,351 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 02:14:31,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 02:14:31,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:32,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:34,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:35,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:37,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:38,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:14:38,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:14:39,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:41,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:43,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:45,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:47,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:47,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:47,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:49,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:50,397 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:14:50,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:51,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:53,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:14:53,359 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:14:53,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:54,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:54,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:55,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:56,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:57,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:14:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:14:59,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:01,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:01,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:03,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:03,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:03,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:04,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:05,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:07,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:08,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:15:08,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:15:09,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:09,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:11,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:13,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:14,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:16,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:17,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:18,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:19,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:19,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:19,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:19,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:19,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:20,470 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:15:22,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:23,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:15:23,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:15:25,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:25,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:32,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:34,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:34,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:36,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:39,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:15:39,122 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:15:42,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:46,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:47,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:47,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,544 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:15:50,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:50,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:52,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:54,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:15:54,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:15:56,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:15:57,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:15:57,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:00,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:01,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:01,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:02,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:05,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:06,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:06,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:06,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:09,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:16:09,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:16:10,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:12,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:15,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:16,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:18,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:20,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,617 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:16:20,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:20,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:21,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:24,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:16:24,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:16:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:25,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:26,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:26,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:26,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:26,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:28,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:30,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:32,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:36,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:36,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:38,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:40,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:16:40,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:16:40,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:42,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:44,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:46,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:47,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:47,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:48,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:50,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:16:50,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:52,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:54,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:55,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:16:55,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:16:56,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:57,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:58,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:16:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:16:59,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:00,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:02,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:02,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:04,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:04,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:06,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:06,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:08,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:10,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:17:10,772 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:17:11,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:11,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:12,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:14,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:14,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:15,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:16,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:17,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:17,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:18,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:19,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:20,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:20,753 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:17:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:22,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:26,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:17:26,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:17:31,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:35,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:35,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:37,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:39,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:41,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:17:41,262 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:17:43,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:46,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:47,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:50,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:17:51,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:51,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:51,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:17:55,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:17:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:17:56,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:17:57,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:01,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:02,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:02,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:03,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:07,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:09,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:11,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:18:11,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:18:11,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:15,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:17,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:17,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:19,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:20,895 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:18:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:22,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:23,455 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 02:18:23,666 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 02:18:23,727 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 02:18:23,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 02:18:23,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:25,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:27,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:18:27,077 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:18:27,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:29,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:31,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:32,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:32,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:33,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:35,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:37,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:39,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:41,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:42,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:18:42,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:18:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:43,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:45,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:47,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:49,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:50,981 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:18:51,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:53,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:55,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:57,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:18:57,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:18:57,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:18:58,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:58,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:18:59,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:01,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:01,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:02,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:02,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:03,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:05,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:07,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:09,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:10,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:10,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:11,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:19:12,797 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:19:12,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:13,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:15,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:21,055 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:19:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:23,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:25,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:28,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:19:28,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:19:29,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:33,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:33,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:35,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:36,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:36,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:40,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:43,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:19:43,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:19:44,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:44,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:46,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:50,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:51,128 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:19:54,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:54,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:56,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:19:57,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:57,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:19:58,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:19:59,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:19:59,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:00,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:02,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:03,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:03,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:06,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:07,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:07,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:08,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:10,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:11,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:11,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:20:13,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:20:14,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:16,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:18,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:18,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:18,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:18,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:18,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:20,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:21,198 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:20:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:23,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:24,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:26,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:28,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:28,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:29,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:20:29,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:20:30,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:30,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:32,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:34,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:36,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:38,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:40,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:42,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:42,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:44,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:20:44,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:20:44,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:46,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:48,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:50,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:51,267 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:20:52,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:54,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:56,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:56,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:57,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:57,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:58,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:20:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:20:59,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:20:59,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:20:59,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:00,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:01,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:02,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:04,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:05,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:06,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:07,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:08,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:09,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:10,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:12,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:15,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:21:15,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:21:17,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:17,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:18,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:19,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:19,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:21,340 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:21:23,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:24,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:27,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:27,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:28,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:30,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:21:30,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:21:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:32,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:32,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:33,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:35,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:37,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:37,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:38,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:39,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:43,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:43,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:45,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:46,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:21:46,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:21:49,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:51,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:21:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:51,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:52,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:52,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:54,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:55,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:57,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:21:58,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:21:58,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:01,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:22:01,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:22:01,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:02,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:03,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:05,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:05,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:07,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:10,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:11,698 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 02:22:11,881 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 02:22:11,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:11,942 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 02:22:12,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 02:22:13,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:15,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:16,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:22:16,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:22:17,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:18,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:18,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:19,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:21,494 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:22:21,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:23,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:23,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:24,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:24,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:25,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:27,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:29,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:22:31,918 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:22:31,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:33,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:35,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:37,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:37,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:39,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:41,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:43,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:45,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:22:47,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:22:47,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:48,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:48,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:49,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:50,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:50,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,565 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:22:51,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:51,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:53,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:54,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:54,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:55,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:57,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:22:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:58,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:22:59,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:01,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:02,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:23:02,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:23:04,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:04,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:11,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:12,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:14,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:15,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:16,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:16,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:17,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:23:17,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:23:21,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:21,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:23:22,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:23,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:23,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:25,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:26,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:28,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:28,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:28,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:28,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:28,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:28,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:28,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:28,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:30,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:32,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:23:32,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:23:32,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:34,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:34,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:36,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:37,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:38,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:40,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:42,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:46,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:48,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:23:48,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:23:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:48,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:51,715 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:23:52,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:53,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:53,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:54,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:23:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:57,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:23:58,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:00,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:02,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:02,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:03,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:24:03,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:24:04,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:05,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:05,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:06,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:08,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:10,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:12,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:14,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:14,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:15,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:16,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:18,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:24:18,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:24:18,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:19,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:19,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:20,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:21,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:21,794 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:24:22,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:22,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:23,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:24,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:26,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:28,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:30,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:32,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:33,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:33,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:24:33,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:24:35,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:37,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:37,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:39,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:41,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:43,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:45,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:45,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:47,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:48,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:48,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:49,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:24:49,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:24:51,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:51,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:51,869 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:24:53,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:54,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:54,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:24:55,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:24:57,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:01,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:01,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:25:04,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:25:08,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:12,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:18,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:18,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:25:19,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:25:20,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:20,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:21,947 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:25:21,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:22,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:23,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:24,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:25,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:27,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:28,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:30,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:31,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:34,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:34,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:25:34,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:25:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:35,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:36,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:38,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:40,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:41,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:44,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:46,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:47,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:47,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:48,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:50,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:25:50,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:25:51,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:52,023 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:25:52,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:54,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:56,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:25:58,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:58,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:25:59,658 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 02:25:59,844 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 02:25:59,915 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 02:26:00,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 02:26:00,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:01,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:02,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:04,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:26:05,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:26:06,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:08,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:10,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:11,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:12,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:14,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:16,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:16,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:18,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:19,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:20,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:20,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:26:20,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:26:21,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:21,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:22,109 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:26:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:24,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:26,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:28,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:30,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:32,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:34,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:36,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:36,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:26:36,107 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:26:38,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:38,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:39,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:41,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:41,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:41,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:43,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:44,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:44,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:45,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:45,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:46,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:47,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:49,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:51,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:26:51,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:26:51,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:26:52,183 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:26:54,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:26:54,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:03,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:06,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:27:06,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:27:07,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:07,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:07,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:11,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:12,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:15,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:18,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:18,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:19,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:19,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:27:21,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:27:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:22,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:27:23,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:23,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:27,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:27,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:29,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:30,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:33,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:35,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:27:37,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:27:39,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:39,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:40,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:45,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:47,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:47,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:47,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:49,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:49,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:52,338 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:27:52,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:27:52,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:27:53,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:54,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:55,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:56,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:57,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:27:58,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:58,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:27:59,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:01,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:02,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:03,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:04,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:05,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:07,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:28:07,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:28:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:08,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:09,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:10,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:11,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:13,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:15,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:17,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:17,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:19,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:21,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,413 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:28:22,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:22,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:23,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:28:23,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:28:23,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:25,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:28,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:29,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:30,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:32,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:34,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:36,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:38,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:38,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:28:38,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:28:40,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:40,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:40,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:41,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:42,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:43,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:44,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:46,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:48,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:52,489 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:28:53,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:28:53,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:28:54,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:28:57,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:28:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:00,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:04,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:04,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:06,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:06,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:08,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:09,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:29:09,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:29:12,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:16,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:20,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:20,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:21,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:22,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:22,567 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:29:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:23,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:24,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:29:24,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:29:26,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:28,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:30,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:32,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:34,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:34,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:36,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:38,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:38,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:38,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:39,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:29:39,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:29:40,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:40,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:40,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:44,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:46,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:46,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:46,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:48,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:48,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 02:29:49,601 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 02:29:49,797 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 02:29:49,869 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 02:29:50,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 02:29:50,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:52,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:52,675 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:29:54,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:55,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:29:55,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:29:56,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:29:58,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:00,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:03,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:05,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:07,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:09,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:10,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:30:10,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:30:11,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:13,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:15,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:17,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:19,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:21,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:22,750 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:30:23,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:25,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:30:25,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:30:27,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:29,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:31,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:33,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:35,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:37,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:39,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:41,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:30:41,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:30:41,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:43,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:45,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:47,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:49,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:51,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:52,827 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:30:53,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:55,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:56,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:30:56,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:30:57,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:30:59,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:01,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:03,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:05,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:07,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:09,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:11,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:11,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:31:11,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:31:13,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:15,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:18,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:20,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:22,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:22,903 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:31:24,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:26,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:31:26,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:31:28,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:30,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:32,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:34,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:36,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:38,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:40,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:42,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:42,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:31:42,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:31:44,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:46,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:48,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:50,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:52,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:52,968 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:31:54,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:56,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:31:57,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:31:57,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:31:58,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:00,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:02,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:04,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:06,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:08,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:10,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:12,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:12,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:32:12,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:32:14,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:16,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:18,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:20,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:22,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:23,042 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:32:24,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:26,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:32:27,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:32:28,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:30,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:32,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:34,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:36,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:38,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:40,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:42,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:43,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:32:43,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:32:45,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:47,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:49,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:51,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:53,119 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:32:53,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:55,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:57,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:32:58,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:32:58,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:32:59,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:01,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:03,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:05,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:07,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:09,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:11,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:13,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:33:13,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:33:15,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:17,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:19,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:21,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:23,194 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:33:23,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:25,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:27,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:28,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:33:28,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:33:29,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:31,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:33,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:35,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:37,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:39,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:41,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:43,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:44,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:33:44,311 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:33:45,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:47,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:49,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:51,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:53,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:53,269 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:33:55,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:57,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:59,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:33:59,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:33:59,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:34:01,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:03,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:05,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:07,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:10,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:12,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:14,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:14,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:34:14,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:34:16,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:18,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:20,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:22,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:23,343 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:34:24,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:26,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:28,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:34:30,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:34:30,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:32,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:34,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:36,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:38,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:40,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:42,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:44,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:45,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:34:45,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:34:46,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:48,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:50,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:52,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:53,416 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:34:54,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:56,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:34:58,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:00,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:00,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:35:00,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:35:02,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:04,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:06,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:08,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:10,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:12,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:14,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:35:15,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:35:16,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:18,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:21,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:23,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:23,491 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:35:25,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:27,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:29,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:31,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:35:31,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:35:31,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:33,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:35,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:37,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:39,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:41,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:43,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:45,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:46,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:35:46,553 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:35:47,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:49,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:51,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:53,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:53,566 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:35:55,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:57,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:35:59,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:01,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:01,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:36:01,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:36:03,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:05,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:07,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:09,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:11,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:13,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:15,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:17,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:36:17,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:36:17,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:19,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:21,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:23,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:23,641 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:36:25,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:27,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:29,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:31,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:32,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:36:32,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:36:33,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:35,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:37,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:39,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:41,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:43,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:45,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:47,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:36:47,829 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:36:48,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:50,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:52,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:53,717 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:36:54,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:56,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:36:58,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:00,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:02,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:03,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:37:03,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:37:04,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:06,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:08,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:10,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:12,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:14,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:16,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:37:18,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:37:18,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:20,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:22,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:23,794 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:37:24,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:26,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:28,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:30,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:32,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:33,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:37:33,573 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:37:34,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:36,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:38,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:40,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:42,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:44,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:46,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:48,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:48,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:37:48,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:37:50,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:52,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:53,872 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:37:54,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:56,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:37:58,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:00,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:02,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:04,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:38:04,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:38:04,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:06,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:08,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:10,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:12,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:14,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:16,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:18,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:19,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:38:19,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:38:20,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:22,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:23,960 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:38:24,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:26,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:28,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:30,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:32,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:34,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:34,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:38:34,997 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:38:36,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:38,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:40,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:42,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:44,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:46,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:49,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:50,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:38:50,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:38:51,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:53,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:54,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:38:55,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:57,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:38:59,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:01,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:03,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:05,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:39:05,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:39:07,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:09,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:11,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:13,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:15,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:17,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:19,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:39:21,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:39:21,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:23,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:24,106 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:39:25,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:27,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:29,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:31,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:33,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:35,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:36,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:39:36,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:39:37,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:39,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:41,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:43,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:45,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:47,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:49,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:51,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:39:51,758 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:39:53,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:54,182 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:39:55,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:57,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:39:59,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:01,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:03,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:05,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:06,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:40:06,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:40:07,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:09,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:11,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:13,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:15,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:17,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:19,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:21,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:40:22,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:40:23,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:24,257 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:40:26,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:28,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:30,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:32,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:34,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:36,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:37,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:40:37,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:40:38,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:40,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:42,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:44,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:46,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:48,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:50,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:52,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:52,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:40:52,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:40:54,330 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:40:54,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:56,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:40:58,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:00,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:02,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:04,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:06,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:41:08,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:41:08,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:10,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:12,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:14,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:16,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:18,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:20,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:22,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:41:23,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:41:24,408 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:41:24,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:26,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:28,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:30,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:32,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:34,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:36,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:38,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:38,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:41:38,664 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:41:40,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:42,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:44,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:46,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:48,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:50,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:52,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:54,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:41:54,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:41:54,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:57,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:41:59,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:01,068 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:42:01,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:03,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:05,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:07,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:09,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:11,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:13,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:14,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:42:14,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:42:15,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:17,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:19,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:21,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:23,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:24,559 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:42:25,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:27,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:42:29,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:42:29,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:31,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:33,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:35,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:37,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:39,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:41,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:43,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:44,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:42:44,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:42:45,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:47,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:49,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:51,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:53,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:54,634 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:42:55,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:57,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:59,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:42:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:42:59,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:43:01,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:03,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:05,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:07,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:09,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:11,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:13,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:43:15,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:43:15,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:17,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:19,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:21,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:23,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:24,711 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:43:25,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:27,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:43:30,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:43:30,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:32,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:34,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:36,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:38,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:40,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:42,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:44,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:45,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:43:45,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:43:46,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:48,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:50,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:52,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:54,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:54,788 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:43:56,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:43:58,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:00,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:00,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:44:00,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:44:02,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:04,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:06,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:08,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:10,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:12,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:14,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:16,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:44:16,346 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:44:16,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:18,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:20,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:22,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:24,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:24,865 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:44:26,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:28,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:30,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:44:31,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:44:32,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:34,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:36,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:38,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:40,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:42,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:44,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:46,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:46,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:44:46,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:44:48,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:50,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:52,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:54,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:54,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:44:56,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:44:58,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:00,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:02,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:45:02,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:45:02,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:04,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:06,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:08,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:10,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:12,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:14,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:16,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:17,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:45:17,416 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:45:18,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:21,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:23,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:25,015 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:45:25,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:27,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:29,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:31,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:32,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:45:32,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:45:33,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:35,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:37,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:39,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:41,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:43,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:45,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:47,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:48,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:45:48,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:45:49,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:51,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:53,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:55,088 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:45:55,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:57,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:45:59,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:01,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:03,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:46:03,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:46:03,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:05,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:07,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:09,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:11,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:13,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:15,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:17,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:18,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:46:18,639 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:46:19,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:21,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:23,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:25,164 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:46:25,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:27,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:29,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:31,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:33,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:46:33,996 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:46:35,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:37,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:39,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:41,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:43,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:45,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:47,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:49,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:46:49,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:46:49,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:51,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:53,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:55,239 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:46:55,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:57,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:46:59,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:01,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:04,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:47:04,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:47:04,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:06,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:08,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:10,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:12,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:14,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:16,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:18,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:19,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:47:19,801 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:47:20,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:22,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:24,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:25,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:47:26,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:28,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:30,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:32,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:34,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:35,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:47:35,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:47:36,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:38,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:40,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:42,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:44,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:46,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:48,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:50,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:47:50,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:47:50,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:52,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:54,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:55,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:47:56,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:47:58,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:00,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:02,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:04,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:48:05,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:48:06,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:08,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:10,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:12,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:14,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:16,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:18,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:20,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:20,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:48:20,864 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:48:22,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:24,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:25,465 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:48:26,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:28,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:30,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:32,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:34,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:48:36,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:48:36,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:38,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:40,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:42,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:45,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:47,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:49,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:51,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:48:51,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:48:51,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:53,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:55,540 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:48:55,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:57,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:48:59,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:01,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:03,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:05,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:49:06,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:49:07,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:09,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:11,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:13,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:15,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:17,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:19,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:21,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:49:22,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:49:23,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:25,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:49:25,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:27,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:29,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:31,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:33,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:35,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:49:37,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:49:37,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:39,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:41,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:43,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:45,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:47,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:49,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:51,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:52,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:49:52,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:49:53,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:55,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:55,691 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:49:57,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:49:59,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:01,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:03,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:06,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:07,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:50:07,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:50:08,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:10,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:12,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:14,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:16,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:18,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:20,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:22,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:23,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:50:23,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:50:24,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:25,768 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:50:26,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:28,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:30,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:32,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:34,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:36,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:38,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:50:38,406 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:50:38,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:40,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:42,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:44,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:46,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:48,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:50,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:52,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:50:53,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:50:54,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:55,844 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:50:56,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:50:58,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:00,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:02,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:04,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:06,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:08,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:08,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:51:08,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:51:10,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:12,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:14,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:16,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:18,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:20,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:22,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:24,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:51:24,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:51:24,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:25,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:51:26,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:28,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:30,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:32,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:34,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:36,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:38,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:39,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:51:39,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:51:40,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:42,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:44,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:46,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:48,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:50,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:52,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:54,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:51:54,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:51:55,998 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:51:56,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:51:58,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:01,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:03,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:05,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:07,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:09,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:52:10,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:52:11,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:13,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:15,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:17,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:19,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:21,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:23,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:25,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:52:25,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:52:25,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:26,074 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:52:27,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:29,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:31,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:33,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:35,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:37,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:39,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:40,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:52:40,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:52:41,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:43,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:45,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:47,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:49,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:51,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:53,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:55,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:55,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:52:55,899 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:52:56,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:52:57,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:52:59,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:01,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:03,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:05,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:07,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:09,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:11,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:53:11,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:53:11,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:13,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:15,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:17,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:19,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:21,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:23,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:25,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:26,223 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:53:26,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:53:26,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:53:27,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:30,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:32,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:34,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:36,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:38,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:40,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:41,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:53:41,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:53:42,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:44,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:46,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:48,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:50,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:52,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:54,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:56,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:53:56,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:53:57,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:53:57,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:53:58,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:00,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:02,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:04,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:06,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:08,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:10,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:12,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:54:12,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:54:12,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:14,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:16,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:18,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:20,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:22,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:24,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:26,374 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:54:26,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:27,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:54:27,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:54:28,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:30,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:32,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:34,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:36,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:38,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:40,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:42,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:42,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:54:42,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:54:44,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:46,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:48,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:50,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:52,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:54,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:56,450 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:54:57,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:54:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:54:58,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:54:59,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:01,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:03,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:05,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:07,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:09,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:11,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:13,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:55:13,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:55:13,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:15,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:17,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:19,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:21,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:23,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:25,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:26,524 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:55:27,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:55:28,813 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:55:29,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:31,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:33,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:35,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:37,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:39,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:41,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:43,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:44,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:55:44,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:55:45,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:47,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:49,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:51,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:53,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:55,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:56,600 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:55:57,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:55:59,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:55:59,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:55:59,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:01,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:03,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:05,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:07,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:09,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:11,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:13,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:14,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:56:14,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:56:15,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:17,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:19,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:21,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:23,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:25,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:26,677 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:56:27,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:29,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:29,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:56:29,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:56:32,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:34,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:36,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:38,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:40,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:42,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:44,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:45,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:56:45,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:56:46,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:48,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:50,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:52,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:54,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:56,753 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:56:56,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:56:58,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:00,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:57:00,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:57:00,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:02,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:04,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:06,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:08,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:10,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:12,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:14,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:15,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:57:15,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:57:16,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:18,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:20,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:22,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:24,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:26,827 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:57:26,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:28,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:30,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:30,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:57:30,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:57:32,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:34,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:36,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:38,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:40,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:42,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:44,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:46,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:57:46,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:57:46,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:49,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:51,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:53,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:55,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:56,905 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:57:57,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:57:59,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:58:01,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:58:01,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:03,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:05,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:07,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:09,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:11,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:13,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:15,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:16,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:58:16,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:58:17,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:19,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:21,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:23,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:25,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:26,979 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:58:27,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:29,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:31,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:32,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:58:32,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:58:33,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:35,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:37,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:39,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:41,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:43,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:45,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:58:47,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:58:47,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:49,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:51,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:53,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:55,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:57,050 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:58:57,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:58:59,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:01,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:02,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:59:02,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:59:03,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:05,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:07,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:09,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:11,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:13,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:15,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:17,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:17,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:59:17,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:59:19,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:22,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:24,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:26,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:27,123 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:59:28,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:30,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:32,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:33,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:59:33,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:59:34,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:36,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:38,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:40,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:42,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:44,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:46,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 02:59:48,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 02:59:48,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:50,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:52,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:54,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:56,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 02:59:57,198 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 02:59:58,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:00,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:02,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:03,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:00:03,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:00:04,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:06,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:08,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:10,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:12,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:14,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:16,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:18,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:19,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:00:19,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:00:20,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:22,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:24,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:26,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:27,274 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:00:28,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:30,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:32,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:00:34,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:00:34,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:36,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:38,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:40,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:42,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:44,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:46,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:48,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:00:49,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:00:50,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:52,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:54,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:56,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:00:57,348 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:00:58,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:00,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:03,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:05,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:01:05,245 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:01:05,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:07,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:09,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:11,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:13,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:15,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:17,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:19,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:01:20,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:01:21,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:23,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:25,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:27,424 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:01:27,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:29,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:31,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:33,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:35,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:01:35,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:01:35,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:37,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:39,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:41,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:43,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:45,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:47,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:49,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:01:50,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:01:51,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:53,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:55,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:57,500 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:01:57,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:01:59,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:01,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:03,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:05,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:06,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:02:06,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:02:07,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:09,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:11,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:13,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:15,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:17,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:19,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:21,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:02:21,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:02:21,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:23,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:25,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:27,573 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:02:28,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:30,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:32,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:34,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:36,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:02:36,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:02:36,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:38,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:40,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:42,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:44,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:46,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:49,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:51,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:02:52,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:02:53,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:55,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:57,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:02:57,649 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:02:59,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:01,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:03,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:05,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:07,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:07,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:03:07,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:03:09,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:11,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:13,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:15,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:17,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:19,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:21,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:22,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:03:22,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:03:23,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:25,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:27,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:27,725 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:03:29,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:31,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:33,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:35,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:37,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:37,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:03:37,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:03:39,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:41,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:43,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:45,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:47,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:49,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:51,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:53,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:53,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:03:53,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:03:55,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:57,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:03:57,801 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:03:59,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:01,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:03,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:05,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:07,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:08,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:04:08,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:04:09,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:11,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:13,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:15,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:17,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:19,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:21,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:04:23,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:04:24,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:26,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:27,875 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:04:28,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:30,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:32,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:34,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:36,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:38,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:39,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:04:39,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:04:40,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:42,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:44,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:46,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:48,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:50,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:52,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:54,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:54,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:04:54,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:04:56,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:04:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:04:58,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:00,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:02,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:04,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:06,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:08,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:09,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:05:09,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:05:10,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:12,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:14,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:16,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:18,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:20,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:22,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:24,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:24,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:05:24,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:05:26,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:28,027 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:05:28,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:30,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:32,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:34,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:36,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:38,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:40,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:05:40,082 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:05:42,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:44,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:46,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:48,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:50,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:52,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:54,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:55,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:05:55,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:05:56,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:58,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:05:58,102 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:06:00,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:02,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:04,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:07,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:09,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:10,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:06:10,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:06:11,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:13,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:15,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:17,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:19,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:21,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:23,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:25,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:25,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:06:25,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:06:27,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:28,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:06:29,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:31,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:33,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:35,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:37,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:39,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:41,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:06:41,207 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:06:43,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:45,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:47,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:49,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:51,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:53,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:55,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:56,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:06:56,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:06:57,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:06:58,250 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:06:59,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:01,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:03,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:05,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:07,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:09,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:11,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:07:11,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:07:13,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:15,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:17,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:19,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:21,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:23,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:25,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:26,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:07:26,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:07:27,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:28,327 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:07:29,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:31,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:33,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:35,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:37,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:39,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:41,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:07:42,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:07:43,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:45,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:47,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:49,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:51,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:53,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:56,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:57,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:07:57,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:07:58,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:07:58,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:08:00,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:02,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:04,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:06,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:08,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:10,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:12,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:08:12,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:08:14,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:16,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:18,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:20,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:22,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:24,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:26,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:08:28,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:08:28,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:28,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:08:30,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:32,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:34,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:36,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:38,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:40,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:42,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:43,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:08:43,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:08:44,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:46,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:48,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:50,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:52,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:54,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:56,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:58,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:08:58,542 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:08:58,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:08:58,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:09:00,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:02,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:04,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:06,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:08,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:11,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:13,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:09:13,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:09:15,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:17,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:19,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:21,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:23,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:25,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:27,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:28,610 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:09:29,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:09:29,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:09:29,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:31,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:33,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:35,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:37,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:39,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:41,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:43,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:44,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:09:44,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:09:45,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:47,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:49,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:51,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:53,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:55,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:57,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:58,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:09:59,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:09:59,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:09:59,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:10:01,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:03,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:05,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:07,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:09,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:11,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:13,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:14,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:10:14,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:10:15,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:17,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:19,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:21,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:23,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:25,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:27,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:28,754 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:10:29,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:30,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:10:30,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:10:31,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:33,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:36,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:38,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:40,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:42,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:44,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:45,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:10:45,542 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:10:46,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:48,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:50,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:52,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:54,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:56,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:58,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:10:58,825 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:11:00,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:00,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:11:00,853 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:11:02,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:04,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:06,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:08,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:10,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:12,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:14,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:16,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:11:16,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:11:16,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:18,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:20,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:22,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:24,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:26,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:28,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:28,896 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:11:30,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:31,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:11:31,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:11:32,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:34,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:36,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:38,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:40,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:42,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:11:46,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:11:46,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:11:58,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:12:01,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:12:01,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:12:17,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:12:17,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:12:29,031 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:12:32,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:12:32,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:12:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:12:42,971 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 03:12:43,017 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 03:12:43,093 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 03:12:43,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 03:12:45,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:12:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:12:47,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:12:51,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:12:59,138 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:13:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:13:02,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:13:05,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:13:18,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:13:18,106 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:13:29,215 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:13:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:13:33,335 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:13:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:13:48,594 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:13:59,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:14:03,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:14:03,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:14:19,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:14:19,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:14:29,368 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:14:34,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:14:34,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:14:49,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:14:49,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:14:59,447 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:15:04,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:15:04,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:15:20,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:15:20,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:15:29,523 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:15:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:15:35,238 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:15:50,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:15:50,467 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:15:59,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:16:05,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:16:05,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:16:20,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:16:20,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:16:29,684 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:16:36,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:16:36,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:16:51,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:16:51,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:16:59,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:16:59,762 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:17:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:00,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:05,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:06,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:17:06,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:17:07,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:08,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:08,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:09,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:10,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:10,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:11,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:13,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:13,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:14,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:15,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:17,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:19,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:21,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:21,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:21,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:17:21,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:17:23,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:25,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:26,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:27,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:29,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:29,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:17:31,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:33,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:35,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:37,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:17:37,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:17:37,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:39,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:40,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:41,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:43,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:45,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:45,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:46,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:47,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:48,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:49,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:17:52,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:17:52,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:17:53,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:53,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:17:59,907 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:18:00,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:04,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:18:07,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:18:08,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:09,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:12,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:16,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:18,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:20,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:22,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:22,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:18:22,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:18:26,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:28,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:28,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:29,976 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:18:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:30,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:31,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:32,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:34,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:35,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:36,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:38,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:18:38,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:18:38,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:38,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:42,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:43,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:43,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:44,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:45,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:46,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:48,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:48,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:50,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:51,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:52,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:53,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:18:53,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:18:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:53,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:54,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:56,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:18:56,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:56,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:18:58,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:00,051 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:19:00,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:02,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:04,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:06,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:08,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:08,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:08,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:19:08,681 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:19:09,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:09,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:10,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:11,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:11,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:12,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:13,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:14,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:16,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:18,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:18,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:18,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:20,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:22,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:19:23,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:19:24,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:26,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:28,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:28,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:28,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:30,127 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:19:30,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:32,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:32,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:32,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:34,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:36,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:38,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:38,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:38,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:39,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:19:39,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:19:40,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:42,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:42,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:43,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:44,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:46,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:19:54,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:19:54,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:19:55,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:55,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:19:56,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,203 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:20:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:00,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:04,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:05,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:05,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:07,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:09,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:09,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:20:09,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:20:13,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:17,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:20,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:20,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:21,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:23,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:23,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:23,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:25,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:20:25,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:20:27,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:29,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:30,279 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:20:32,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:33,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:35,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:38,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:38,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:39,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:20:40,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:20:40,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:40,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:41,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:42,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:42,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:43,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:47,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:47,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:48,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:49,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:49,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:50,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:50,241 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 03:20:50,459 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 03:20:50,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 03:20:50,533 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 03:20:51,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:52,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:53,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:55,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:20:55,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:20:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:20:57,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:20:59,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,555 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:21:00,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:00,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:01,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:03,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:05,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:07,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:08,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:09,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:21:10,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:21:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:10,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:11,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:13,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:14,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:14,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:14,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:14,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:15,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:17,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:18,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:19,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:21,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:23,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:23,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:25,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:25,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:26,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:21:26,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:21:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:27,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:29,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:30,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:21:31,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:31,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:31,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:33,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:35,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:37,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:39,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:40,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:40,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:41,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:21:41,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:21:42,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:45,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:45,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:50,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:56,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:21:56,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:21:56,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:21:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:21:58,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:00,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:00,689 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:22:01,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:01,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:04,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:08,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:08,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:22:12,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:22:12,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:14,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:17,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:18,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:20,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:20,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:22,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:24,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:25,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:26,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:27,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:22:27,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:22:30,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:30,762 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:22:31,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:31,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:32,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:33,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:36,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:37,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:37,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:38,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:38,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:40,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:42,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:22:42,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:22:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:43,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:44,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:45,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:45,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:48,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:49,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:49,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:50,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:52,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:53,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:53,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:54,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:56,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:22:57,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:22:57,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:22:58,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:58,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:59,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:59,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:59,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:59,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:22:59,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:00,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:00,835 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:23:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:02,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:02,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:04,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:06,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:07,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:07,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:08,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:09,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:10,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:12,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:12,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:13,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:23:13,125 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:23:13,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:13,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:14,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:16,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:17,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:17,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:18,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:19,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:20,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:22,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:24,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:26,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:26,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:23:28,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:23:28,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:30,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:30,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:30,908 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:23:33,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:35,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:37,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:23:43,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:23:44,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:47,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:51,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:57,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:23:58,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:23:58,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:00,986 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:24:01,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:03,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:24:03,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:24:04,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:07,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:11,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:15,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:17,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:18,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:24:18,890 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:24:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:20,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:21,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:23,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:29,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:30,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:31,057 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:24:31,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:34,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:24:34,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:24:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:34,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:35,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:37,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:37,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:39,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:40,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:40,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:41,698 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 03:24:41,904 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 03:24:41,975 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 03:24:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:42,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 03:24:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:43,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:45,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:47,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:48,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:24:49,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:24:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:49,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:51,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:51,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:53,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:55,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:55,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:57,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:24:58,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:58,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:24:59,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:00,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:00,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:01,138 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:25:01,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:02,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:03,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:04,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:25:04,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:25:06,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:07,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:08,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:08,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:09,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:10,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:12,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:13,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:13,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:14,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:16,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:17,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:17,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:18,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:19,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:19,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:25:19,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:25:20,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:22,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:23,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:23,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:24,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:24,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:26,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:28,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:29,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:29,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:30,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:31,212 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:25:32,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:35,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:25:35,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:25:40,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:46,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:46,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:25:50,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:25:50,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:54,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:25:54,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:25:58,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:00,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:01,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:26:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:04,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:05,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:26:05,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:26:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:08,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:10,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:14,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:16,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:18,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:20,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:26:20,856 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:26:22,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:24,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:24,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:26,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:27,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:29,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:30,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:31,367 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:26:31,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:33,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:33,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:35,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:36,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:26:36,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:26:36,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:36,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:37,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:39,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:40,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:40,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:41,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:42,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:42,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:43,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:44,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:45,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:45,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:46,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:47,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:49,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:49,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:51,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:26:51,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:26:51,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:51,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:51,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:52,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:53,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:54,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:54,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:55,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:57,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:26:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:26:59,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:01,443 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:27:01,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:03,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:03,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:04,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:05,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:06,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:27:06,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:27:07,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:07,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:09,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:11,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:11,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:12,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:13,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:15,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:17,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:18,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:19,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:21,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:21,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:27:21,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:27:22,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:22,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:23,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:25,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:31,516 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:27:31,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:37,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:38,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:27:38,827 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:27:41,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:45,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:49,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:53,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:27:54,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:27:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:54,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:27:55,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:27:59,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:01,587 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:28:03,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:05,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:07,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:07,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:08,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:09,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:28:09,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:28:09,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:11,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:13,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:16,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:17,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:18,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:19,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:21,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:23,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:23,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:28:24,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:28:24,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:24,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:24,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:24,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:25,589 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 03:28:25,733 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 03:28:25,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 03:28:25,797 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 03:28:26,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:26,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:27,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:28,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:29,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:30,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:30,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:31,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:31,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:28:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:33,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:34,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:35,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:36,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:36,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:37,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:38,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:38,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:39,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:39,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:28:39,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:28:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:40,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:41,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:42,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:42,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:43,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:44,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:45,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:47,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:47,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:47,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:48,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:49,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:51,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:53,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:53,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:55,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:28:55,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:28:55,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:57,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:58,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:58,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:28:59,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:28:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:00,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:01,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:01,884 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:29:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:02,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:03,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:04,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:04,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:05,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:06,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:08,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:29:10,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:29:10,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:10,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:12,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:12,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:14,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:16,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:24,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:24,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:29:25,589 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:29:26,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:30,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:31,956 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:29:34,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:38,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:40,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:29:40,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:29:41,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:41,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:42,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:43,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:46,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:48,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:48,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:52,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:54,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:56,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:29:56,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:29:58,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:29:58,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:29:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:00,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:02,033 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:30:03,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:03,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:04,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:05,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:05,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:06,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:07,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:07,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:10,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:11,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:30:11,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:30:12,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:14,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:15,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:15,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:16,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:17,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:17,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:17,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:18,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:20,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:22,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:23,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:23,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:24,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:26,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:30:26,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:30:26,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:28,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:29,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:29,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:30,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,109 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:30:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:32,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:33,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:33,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:34,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:36,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:37,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:37,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:38,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:39,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:40,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:30:41,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:30:42,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:42,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:43,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:44,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:45,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:45,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:46,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:48,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:50,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:51,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:52,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:54,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:54,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:55,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:56,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:57,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:30:57,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:30:58,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:30:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:30:59,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:00,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:01,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,187 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:31:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:02,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:03,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:03,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:04,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:06,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:07,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:08,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:12,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:31:12,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:31:19,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:23,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:31:27,578 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:31:27,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:31,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:32,258 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:31:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:32,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:35,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:38,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:38,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:39,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:41,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:31:42,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:31:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:44,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:45,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:47,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:51,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:52,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:53,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:57,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:31:58,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:31:58,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:31:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:31:59,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:02,334 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:32:02,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:02,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:03,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:05,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:05,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:06,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:07,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:09,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:10,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:11,292 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 03:32:11,509 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 03:32:11,577 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 03:32:11,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 03:32:12,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:12,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:12,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:13,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:13,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:32:13,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:32:13,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:15,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:17,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:18,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:19,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:21,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:23,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:23,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:25,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:25,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:26,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:27,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:28,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:32:28,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:32:29,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:30,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:30,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:31,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:32,427 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:32:33,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:34,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:34,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:35,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:36,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:36,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:37,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:39,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:39,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:41,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:43,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:32:44,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:32:44,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:44,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:45,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:46,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:46,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:47,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:49,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:49,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:50,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:50,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:51,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:53,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:53,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:54,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:55,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:56,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:56,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:57,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:32:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:32:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:32:59,338 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:32:59,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:02,499 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:33:02,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:04,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:06,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:06,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:12,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:14,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:33:14,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:33:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:16,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:20,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:24,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:28,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:29,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:33:29,878 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:33:31,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:31,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:32,576 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:33:32,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:36,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:37,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:37,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:38,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:42,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:43,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:43,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:44,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:33:45,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:33:48,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:50,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:52,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:54,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:56,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:58,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:33:59,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:33:59,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:34:00,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:34:00,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:01,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:01,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:02,650 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:34:03,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:03,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:04,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:04,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:06,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:07,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:08,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:10,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:12,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:12,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:15,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:34:15,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:34:16,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:17,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:17,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:18,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:20,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:22,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:23,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:23,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:24,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:25,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:25,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:26,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:27,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:28,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:29,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:29,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:30,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:31,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:34:31,058 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:34:31,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:31,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:32,724 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:34:32,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:33,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:34,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:36,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:36,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:37,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:37,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:38,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:40,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:40,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:41,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:41,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:42,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:44,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:45,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:45,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:46,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:34:46,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:34:46,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:47,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:48,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:49,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:49,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:50,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:51,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:51,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:52,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:54,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:56,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:34:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:34:59,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:01,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:35:01,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:35:02,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:35:07,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:09,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:11,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:35:16,797 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:35:17,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:20,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:21,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:25,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:27,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:31,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:35:32,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:35:32,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:35:35,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:36,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:37,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:40,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:41,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:43,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:45,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:47,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:35:47,447 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:35:49,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:51,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:52,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:53,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:56,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:56,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:57,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:35:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:59,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:35:59,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 03:36:00,554 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 03:36:00,763 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 03:36:00,827 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 03:36:00,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 03:36:01,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:36:02,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:36:02,988 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:36:03,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:05,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:07,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:09,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:11,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:13,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:15,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:17,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:36:17,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:36:17,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:19,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:21,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:23,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:25,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:27,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:29,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:32,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:33,044 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:36:33,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:36:33,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:36:34,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:37,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:39,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:41,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:43,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:45,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:47,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:48,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:36:48,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:36:49,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:51,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:53,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:55,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:57,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:36:59,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:01,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:03,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:03,117 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:37:03,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:37:03,894 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:37:05,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:07,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:09,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:11,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:13,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:15,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:17,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:19,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:19,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:37:19,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:37:21,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:23,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:25,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:27,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:29,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:31,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:33,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:33,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:37:34,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:37:34,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:37:35,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:37,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:39,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:41,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:43,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:45,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:48,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:49,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:37:49,828 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:37:50,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:52,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:54,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:56,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:37:58,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:00,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:02,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:03,271 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:38:04,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:05,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:38:05,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:38:06,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:08,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:10,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:12,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:14,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:16,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:18,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:20,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:20,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:38:20,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:38:22,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:24,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:26,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:28,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:30,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:32,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:33,347 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:38:34,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:35,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:38:35,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:38:36,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:38,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:40,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:42,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:44,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:46,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:48,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:50,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:50,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:38:50,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:38:52,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:54,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:56,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:38:58,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:00,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:02,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:03,421 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:39:04,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:06,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:06,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:39:06,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:39:08,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:10,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:12,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:14,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:16,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:18,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:20,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:21,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:39:21,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:39:22,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:25,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:27,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:29,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:31,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:33,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:33,497 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:39:35,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:36,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:39:36,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:39:37,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:39,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:41,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:43,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:45,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:47,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:49,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:51,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:39:51,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:39:53,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:55,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:57,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:39:59,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:01,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:03,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:03,572 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:40:05,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:07,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:07,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:40:07,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:40:09,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:11,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:13,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:15,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:17,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:19,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:21,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:40:22,491 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:40:23,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:25,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:27,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:29,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:31,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:33,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:33,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:40:35,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:37,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:40:37,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:40:39,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:41,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:43,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:45,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:47,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:49,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:51,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:53,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:53,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:40:53,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:40:56,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:40:58,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:00,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:02,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:03,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:41:04,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:06,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:08,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:41:08,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:41:10,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:12,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:14,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:16,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:18,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:20,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:22,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:23,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:41:23,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:41:24,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:26,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:28,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:30,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:32,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:33,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:41:34,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:36,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:38,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:38,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:41:38,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:41:40,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:42,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:44,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:46,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:48,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:50,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:52,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:54,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:54,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:41:54,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:41:56,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:41:58,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:00,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:02,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:03,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:42:04,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:06,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:08,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:09,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:42:09,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:42:10,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:12,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:14,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:16,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:18,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:20,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:22,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:24,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:42:24,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:42:26,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:28,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:30,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:32,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:33,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:42:34,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:36,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:38,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:40,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:40,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:42:40,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:42:43,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:45,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:47,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:49,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:51,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:53,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:55,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:55,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:42:55,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:42:57,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:42:59,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:01,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:03,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:03,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:43:05,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:07,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:09,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:43:10,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:43:11,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:13,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:15,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:17,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:19,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:21,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:23,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:25,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:26,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:43:26,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:43:27,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:29,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:31,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:33,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:34,054 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:43:35,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:37,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:39,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:41,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:43:41,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:43:43,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:45,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:47,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:49,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:51,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:53,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:55,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:56,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:43:56,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:43:57,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:43:59,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:01,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:03,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:04,129 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:44:05,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:07,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:09,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:11,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:44:11,837 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:44:13,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:16,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:18,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:20,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:22,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:24,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:26,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:44:27,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:44:28,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:30,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:32,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:34,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:34,203 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:44:36,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:38,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:40,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:42,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:42,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:44:42,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:44:44,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:46,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:48,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:50,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:52,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:54,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:56,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:44:57,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:44:57,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:44:58,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:00,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:02,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:04,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:04,279 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:45:06,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:08,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:10,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:12,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:12,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:45:12,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:45:14,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:16,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:18,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:20,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:22,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:24,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:26,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:45:28,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:45:28,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:30,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:32,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:34,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:34,338 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:45:36,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:38,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:40,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:42,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:43,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:45:43,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:45:44,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:46,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:48,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:51,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:53,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:55,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:57,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:45:58,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:45:58,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:45:59,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:01,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:03,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:04,411 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:46:05,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:07,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:09,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:11,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:13,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:13,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:46:13,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:46:15,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:17,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:19,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:21,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:23,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:25,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:27,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:29,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:29,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:46:29,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:46:31,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:33,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:34,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:46:35,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:37,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:39,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:41,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:43,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:44,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:46:44,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:46:45,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:47,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:49,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:51,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:53,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:55,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:57,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:59,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:46:59,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:46:59,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:47:01,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:03,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:04,563 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:47:05,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:07,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:09,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:11,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:13,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:47:15,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:47:15,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:17,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:20,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:22,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:24,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:26,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:28,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:30,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:30,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:47:30,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:47:32,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:34,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:34,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:47:36,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:38,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:40,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:42,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:44,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:45,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:47:45,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:47:46,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:48,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:50,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:52,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:54,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:56,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:47:58,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:00,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:01,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:48:01,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:48:02,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:04,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:04,715 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:48:06,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:08,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:10,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:12,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:14,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:16,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:16,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:48:16,329 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:48:18,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:20,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:22,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:24,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:26,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:28,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:30,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:31,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:48:31,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:48:32,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:34,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:34,778 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:48:36,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:38,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:40,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:42,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:44,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:46,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:46,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:48:46,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:48:48,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:50,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:52,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:55,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:57,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:48:59,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:01,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:02,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:49:02,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:49:03,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:04,854 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:49:05,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:07,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:09,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:11,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:13,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:15,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:17,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:49:17,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:49:19,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:21,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:23,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:25,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:27,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:29,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:31,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:32,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:49:32,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:49:33,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:34,931 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:49:35,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:37,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:39,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:41,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:43,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:45,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:47,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:47,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:49:47,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:49:49,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:51,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:53,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:55,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:57,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:49:59,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:01,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:03,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:03,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:50:03,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:50:05,008 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:50:05,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:07,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:09,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:11,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:13,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:15,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:17,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:18,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:50:18,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:50:19,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:21,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:23,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:25,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:27,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:29,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:31,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:33,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:34,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:50:34,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:50:35,076 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:50:35,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:37,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:39,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:41,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:44,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:46,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:48,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:49,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:50:49,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:50:50,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:52,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:54,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:56,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:50:58,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:00,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:02,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:04,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:51:04,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:51:05,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:51:06,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:08,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:10,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:12,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:14,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:16,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:18,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:51:19,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:51:20,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:22,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:24,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:26,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:28,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:30,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:32,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:34,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:51:35,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:51:35,424 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:51:36,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:38,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:40,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:42,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:44,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:46,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:48,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:50,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:50,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:51:50,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:51:52,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:54,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:56,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:51:58,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:00,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:02,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:04,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:05,268 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:52:05,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:52:05,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:52:06,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:08,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:10,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:13,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:15,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:17,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:19,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:52:21,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:52:21,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:23,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:25,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:27,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:29,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:31,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:33,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:35,341 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:52:35,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:36,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:52:36,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:52:37,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:39,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:41,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:43,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:45,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:47,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:49,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:51,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:52:51,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:52:53,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:55,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:57,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:52:59,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:01,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:03,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:05,414 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:53:06,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:07,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:53:07,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:53:08,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:10,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:12,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:14,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:16,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:18,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:20,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:22,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:53:22,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:53:22,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:24,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:26,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:28,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:30,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:32,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:34,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:35,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:53:36,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:37,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:53:37,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:53:38,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:40,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:42,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:44,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:46,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:48,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:50,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:52,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:52,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:53:52,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:53:54,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:56,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:53:58,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:00,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:02,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:04,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:05,536 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:54:06,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:54:08,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:54:08,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:10,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:12,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:14,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:16,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:19,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:21,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:23,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:23,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:54:23,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:54:25,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:27,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:29,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:31,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:33,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:35,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:35,612 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:54:37,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:54:38,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:54:39,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:41,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:43,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:45,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:47,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:49,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:51,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:53,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:54,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:54:54,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:54:55,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:57,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:54:59,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:01,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:03,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:05,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:05,686 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:55:07,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:55:09,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:55:09,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:11,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:13,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:15,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:17,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:19,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:21,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:23,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:55:24,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:55:25,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:27,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:29,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:31,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:33,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:35,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:35,750 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:55:37,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:39,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:39,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:55:39,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:55:41,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:43,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:45,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:47,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:50,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:52,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:54,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:55,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:55:55,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:55:56,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:55:58,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:00,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:02,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:04,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:05,823 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:56:06,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:08,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:10,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:56:10,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:56:10,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:12,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:14,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:16,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:18,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:20,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:22,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:24,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:56:25,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:56:26,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:28,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:30,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:32,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:34,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:35,900 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:56:36,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:38,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:40,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:40,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:56:40,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:56:42,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:44,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:46,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:48,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:50,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:52,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:54,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:56:56,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:56:56,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:56:58,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:00,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:02,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:04,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:05,962 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:57:06,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:08,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:10,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:11,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:57:11,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:57:12,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:14,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:16,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:18,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:21,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:23,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:25,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:26,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:57:26,886 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:57:27,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:29,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:31,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:33,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:35,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:36,035 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:57:37,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:39,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:41,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:42,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:57:42,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:57:43,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:45,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:47,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:49,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:51,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:53,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:55,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:57,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:57:57,387 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:57:57,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:57:59,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:01,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:03,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:05,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:06,110 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:58:07,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:09,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:11,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:12,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:58:12,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:58:13,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:15,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:17,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:19,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:21,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:23,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:25,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:27,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:58:27,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:58:29,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:31,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:33,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:35,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:36,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:58:37,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:39,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:41,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:58:43,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:58:43,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:45,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:47,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:49,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:51,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:53,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:55,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:57,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:58:58,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:58:58,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:58:59,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:01,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:03,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:05,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:06,257 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:59:07,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:10,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:12,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:13,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:59:13,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:59:14,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:16,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:18,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:20,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:22,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:24,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:26,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:28,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:29,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:59:29,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:59:30,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:32,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:34,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:36,330 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 03:59:36,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:38,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:40,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:42,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:44,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:59:44,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 03:59:44,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:46,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:48,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:50,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:52,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:54,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:56,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:58,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 03:59:59,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 03:59:59,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:00:00,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:02,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:04,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:06,406 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:00:06,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:08,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:10,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:12,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:14,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:00:14,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:00:16,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:18,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:20,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:22,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:24,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:26,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:28,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:30,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:00:30,227 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:00:30,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:33,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:35,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:36,479 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:00:37,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:39,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:41,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:43,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:45,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:00:45,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:00:47,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:49,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:51,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:53,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:55,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:57,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:00:59,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:01,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:01:01,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:01:01,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:03,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:05,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:06,549 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:01:07,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:09,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:11,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:13,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:15,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:16,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:01:16,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:01:17,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:19,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:21,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:23,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:25,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:27,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:29,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:31,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:01:31,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:01:33,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:35,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:36,625 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:01:37,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:39,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:41,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:43,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:45,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:47,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:01:47,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:01:47,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:49,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:51,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:53,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:55,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:57,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:01:59,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:01,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:02,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:02:02,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:02:03,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:05,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:06,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:02:07,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:09,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:11,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:13,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:15,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:17,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:02:17,627 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:02:17,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:19,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:21,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:23,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:25,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:27,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:29,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:31,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:32,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:02:32,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:02:33,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:35,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:36,774 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:02:38,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:40,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:42,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:44,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:46,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:48,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:02:48,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:02:48,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:50,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:52,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:54,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:56,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:02:58,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:00,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:02,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:03,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:03:03,467 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:03:04,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:06,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:06,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:03:08,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:10,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:12,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:14,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:16,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:18,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:18,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:03:18,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:03:20,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:22,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:24,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:26,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:28,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:30,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:32,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:34,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:03:34,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:03:34,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:36,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:36,923 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:03:38,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:40,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:42,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:44,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:46,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:48,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:03:49,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:03:50,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:52,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:54,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:56,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:03:58,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:00,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:02,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:04,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:04,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:04:04,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:04:06,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:06,998 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:04:08,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:10,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:12,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:14,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:16,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:18,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:20,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:04:20,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:04:20,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:22,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:25,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:27,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:29,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:31,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:33,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:04:35,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:04:35,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:37,072 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:04:37,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:39,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:41,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:43,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:45,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:47,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:49,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:50,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:04:50,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:04:51,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:53,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:55,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:57,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:04:59,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:01,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:03,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:05,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:05:05,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:05:07,147 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:05:07,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:09,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:11,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:13,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:15,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:17,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:19,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:21,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:05:21,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:05:21,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:23,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:25,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:27,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:29,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:31,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:33,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:35,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:36,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:05:36,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:05:37,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:05:37,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:39,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:41,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:43,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:45,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:47,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:49,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:51,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:05:52,293 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:05:53,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:55,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:57,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:05:59,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:01,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:03,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:05,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:07,296 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:06:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:06:07,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:06:07,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:09,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:11,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:13,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:15,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:17,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:19,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:21,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:06:22,934 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:06:23,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:25,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:27,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:30,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:32,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:34,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:36,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:37,371 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:06:38,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:06:38,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:06:38,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:40,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:42,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:44,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:46,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:48,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:50,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:52,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:54,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:06:54,058 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:06:54,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:56,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:06:58,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:00,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:02,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:04,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:06,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:07,446 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:07:08,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:09,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:07:09,403 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:07:10,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:12,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:14,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:16,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:18,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:20,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:22,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:07:24,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:07:24,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:26,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:28,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:30,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:32,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:34,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:36,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:37,523 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:07:38,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:40,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:07:40,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:07:40,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:42,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:44,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:46,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:48,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:51,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:53,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:55,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:07:55,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:07:55,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:57,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:07:59,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:01,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:03,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:05,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:07,597 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:08:07,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:09,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:10,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:08:10,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:08:11,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:13,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:15,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:17,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:19,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:21,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:23,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:25,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:25,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:08:25,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:08:27,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:29,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:31,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:33,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:35,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:37,671 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:08:37,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:39,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:08:41,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:08:41,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:43,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:45,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:47,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:49,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:51,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:53,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:55,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:56,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:08:56,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:08:57,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:08:59,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:01,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:03,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:05,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:07,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:09:07,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:09,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:11,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:09:11,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:09:13,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:15,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:17,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:19,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:21,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:23,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:25,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:27,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:09:27,035 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:09:27,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:30,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:32,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:34,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:36,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:37,818 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:09:38,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:40,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:42,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:09:42,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:09:42,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:44,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:46,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:48,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:50,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:52,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:54,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:56,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:09:57,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:09:57,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:09:58,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:00,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:02,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:04,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:06,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:07,895 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:10:08,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:10,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:12,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:10:12,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:10:12,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:14,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:16,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:18,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:20,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:22,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:24,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:26,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:28,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:10:28,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:10:28,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:30,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:32,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:34,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:36,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:37,969 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:10:38,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:40,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:42,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:43,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:10:43,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:10:44,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:46,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:48,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:50,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:52,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:54,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:56,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:10:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:10:58,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:10:58,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:00,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:02,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:04,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:07,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:08,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:11:09,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:11,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:13,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:13,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:11:13,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:11:15,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:17,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:19,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:21,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:23,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:25,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:27,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:11:29,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:11:29,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:31,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:33,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:35,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:37,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:38,119 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:11:39,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:41,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:43,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:44,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:11:44,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:11:45,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:47,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:49,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:51,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:53,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:55,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:57,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:11:59,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:11:59,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:11:59,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:01,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:03,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:05,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:07,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:08,193 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:12:09,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:11,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:13,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:14,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:12:14,946 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:12:15,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:18,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:20,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:22,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:24,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:26,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:28,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:30,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:12:30,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:12:30,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:32,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:34,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:36,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:38,268 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:12:38,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:40,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:42,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:44,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:45,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:12:45,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:12:46,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:48,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:50,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:52,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:54,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:56,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:12:58,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:00,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:13:00,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:13:00,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:02,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:04,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:06,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:08,344 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:13:08,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:10,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:12,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:14,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:16,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:13:16,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:13:16,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:18,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:20,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:22,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:24,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:26,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:29,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:13:31,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:13:31,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:33,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:35,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:37,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:38,421 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:13:39,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:41,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:43,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:45,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:13:46,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:13:47,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:49,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:51,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:53,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:55,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:57,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:13:59,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:14:01,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:14:01,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:03,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:05,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:07,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:08,495 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:14:09,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:11,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:13,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:15,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:17,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:14:17,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:14:17,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:19,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:21,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:23,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:25,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:27,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:29,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:31,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:32,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:14:32,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:14:33,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:35,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:37,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:38,571 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:14:40,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:42,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:44,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:46,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:47,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:14:47,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:14:48,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:50,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:52,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:54,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:56,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:14:58,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:00,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:02,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:02,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:15:02,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:15:04,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:06,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:08,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:15:08,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:10,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:12,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:14,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:16,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:18,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:15:18,346 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:15:18,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:20,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:22,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:24,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:26,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:28,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:30,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:32,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:33,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:15:33,632 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:15:34,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:36,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:38,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:15:38,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:40,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:42,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:44,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:46,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:48,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:15:48,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:15:49,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:51,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:53,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:55,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:57,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:15:59,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:01,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:03,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:04,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:16:04,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:16:05,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:07,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:08,792 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:16:09,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:11,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:13,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:15,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:17,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:16:19,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:16:19,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:21,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:23,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:25,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:27,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:29,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:31,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:33,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:34,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:16:34,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:16:35,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:37,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:38,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:16:39,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:41,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:43,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:45,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:47,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:49,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:16:50,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:16:51,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:53,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:56,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:16:58,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:00,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:02,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:04,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:05,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:17:05,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:17:06,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:08,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:08,937 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:17:10,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:12,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:15,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:17,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:19,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:17:20,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:17:21,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:23,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:25,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:27,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:29,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:31,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:33,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:35,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:35,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:17:35,934 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:17:37,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:39,007 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:17:39,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:41,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:43,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:45,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:47,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:49,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:51,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:51,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:17:51,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:17:53,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:17:55,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:18:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:18:06,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:18:09,078 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:18:22,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:18:22,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:18:37,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:18:37,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:18:39,153 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:18:52,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:18:52,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:18:55,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:18:55,556 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 04:18:55,608 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 04:18:55,684 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 04:18:56,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 04:18:57,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:19:03,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:19:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:19:07,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:19:09,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:19:19,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:19:23,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:19:23,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:19:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:19:38,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:19:39,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:19:53,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:19:53,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:20:08,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:20:08,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:20:09,391 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:20:24,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:20:24,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:20:39,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:20:39,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:20:39,669 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:20:54,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:20:54,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:21:09,538 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:21:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:21:09,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:21:25,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:21:25,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:21:39,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:21:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:21:40,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:21:55,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:21:55,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:22:09,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:22:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:22:10,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:22:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:22:26,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:22:39,760 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:22:41,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:22:41,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:22:56,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:22:56,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:23:09,832 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:23:10,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:11,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:23:11,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:23:12,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:12,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:14,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:15,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:16,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:18,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:20,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:20,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:22,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:24,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:26,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:26,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:26,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:23:27,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:23:27,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:27,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:28,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:29,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:30,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:31,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:31,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:32,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:32,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:34,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:35,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:36,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:36,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:37,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:38,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:39,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:39,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:23:40,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:41,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:42,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:23:42,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:23:42,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:44,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:45,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:46,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:48,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:49,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:52,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:53,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:53,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:54,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:54,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:56,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:23:57,677 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:23:57,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:57,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:58,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:23:59,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:23:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:00,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:02,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:09,971 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:24:12,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:12,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:12,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:24:12,907 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:24:18,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:22,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:23,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:23,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:26,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:28,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:24:28,163 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:24:30,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:33,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:33,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:34,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:36,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:40,042 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:24:40,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:43,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:24:43,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:24:44,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:44,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:44,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:45,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:46,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:50,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:52,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:53,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:53,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:24:56,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:58,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:24:58,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:24:58,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:25:00,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:01,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:04,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:06,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:06,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:06,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:08,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:08,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:10,115 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:25:10,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:11,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:13,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:25:13,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:25:14,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:16,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:18,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:18,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:20,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:20,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:22,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:22,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:23,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:23,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:24,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:26,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:28,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:29,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:29,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:25:29,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:25:30,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:30,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:31,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:32,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:33,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:34,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:36,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:37,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:25:40,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:41,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:42,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:42,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:42,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:42,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:42,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:42,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:42,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:43,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:44,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:25:44,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:25:45,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:45,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:47,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:48,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:49,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:50,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:50,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:51,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:52,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:53,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:53,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:55,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:55,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:25:57,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:59,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:25:59,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:25:59,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:26:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:00,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:07,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:10,249 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:26:12,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:13,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:14,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:26:14,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:26:15,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:17,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:21,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:23,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:23,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:25,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:29,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:30,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:26:30,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:26:30,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:32,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:33,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:37,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:39,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:40,321 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:26:43,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:45,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:45,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:26:45,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:26:47,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:49,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:51,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:55,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:57,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:26:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:57,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:26:59,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:00,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:27:00,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:27:01,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:02,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:02,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:03,034 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 04:27:03,188 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 04:27:03,248 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 04:27:04,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 04:27:04,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:04,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:04,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:05,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:07,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:09,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:10,464 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:27:11,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:13,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:14,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:16,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:27:16,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:27:17,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:17,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:19,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:21,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:23,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:23,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:25,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:25,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:27,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:28,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:29,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:30,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:31,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:31,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:27:31,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:27:33,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:33,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:35,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:37,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:37,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:37,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:38,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:39,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,531 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:27:40,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:40,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:41,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:43,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:45,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:46,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:27:46,594 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:27:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:46,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:47,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:48,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:48,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:49,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:50,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:50,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:51,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:27:53,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:53,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:27:55,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:01,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:28:01,980 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:28:04,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:07,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:07,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:08,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:10,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:28:12,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:16,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:17,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:28:17,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:28:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:19,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:20,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:24,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:24,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:24,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:28,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:29,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:29,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:30,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:32,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:28:32,835 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:28:34,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:36,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:36,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:38,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:40,679 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:28:42,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:44,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:45,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:45,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:46,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:48,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:28:48,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:28:49,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:49,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:49,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:49,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:49,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:49,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:50,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:52,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:54,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:54,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:58,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:28:59,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:28:59,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:00,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:02,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:03,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:29:03,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:29:04,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:06,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:08,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:08,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:08,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:10,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:10,755 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:29:11,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:11,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:12,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:14,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:14,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:15,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:16,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:18,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:18,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:29:18,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:29:19,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:20,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:22,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:23,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:24,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:25,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:25,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:26,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:26,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:27,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:28,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:28,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:30,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:30,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:31,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:32,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:32,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:29:33,969 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:29:34,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:34,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:35,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:36,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:36,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:38,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:40,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:40,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:40,830 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:29:41,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:41,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:42,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:44,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:46,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:47,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:29:49,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:29:52,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:29:53,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:53,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:55,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:29:58,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:02,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:04,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:30:04,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:30:05,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:05,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:06,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:07,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:10,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:10,906 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:30:11,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:14,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:16,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:20,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:30:20,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:30:20,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:21,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:21,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:22,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:26,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:28,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:28,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:30,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:32,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:34,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:30:35,366 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:30:38,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:40,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:40,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:40,980 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:30:42,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:44,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:45,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:45,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:46,592 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 04:30:46,798 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 04:30:46,861 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 04:30:47,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 04:30:47,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:48,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:50,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:50,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:30:50,686 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:30:52,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:52,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:53,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:53,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:54,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:56,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:30:57,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:57,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:30:59,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:01,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:01,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:03,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:05,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:31:05,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:31:07,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:07,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:08,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:08,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:09,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:11,066 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:31:11,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:12,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:13,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:15,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:17,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:19,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:19,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:19,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:20,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:21,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:31:21,218 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:31:21,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:23,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:24,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:25,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:25,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:26,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:27,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:28,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:29,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:30,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:31,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:32,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:33,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:35,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:36,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:31:36,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:31:36,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:37,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:39,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:41,141 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:31:45,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:45,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:47,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:31:51,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:31:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:52,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:31:53,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:31:57,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:00,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:00,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:01,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:05,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:05,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:32:07,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:32:08,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:08,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:09,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:11,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,224 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:32:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:11,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:15,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:17,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:21,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:22,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:22,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:32:22,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:32:23,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:27,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:29,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:29,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:29,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:33,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:35,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:37,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:37,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:32:37,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:32:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:39,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:40,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:40,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:41,298 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:32:43,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:44,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:45,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:47,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:49,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:50,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:51,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:32:53,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:32:53,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:55,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:55,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:57,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:32:58,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:58,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:32:59,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:01,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:03,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:04,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:06,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:07,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:07,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:08,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:33:08,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:33:08,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:10,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:10,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:11,369 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:33:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:14,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:16,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:16,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:17,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:17,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:18,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:19,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:20,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:21,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:33:23,528 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:33:23,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:23,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:24,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:26,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:26,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:28,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:28,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:30,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:32,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:34,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:33:38,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:33:41,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:33:42,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:43,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:48,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:51,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:52,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:33:54,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:33:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:56,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:33:59,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:33:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:00,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:01,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:04,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:06,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:34:09,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:34:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:10,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:11,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:34:11,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:12,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:12,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:16,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:18,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:22,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:24,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:34:24,848 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:34:25,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:26,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:27,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:29,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:30,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:32,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:34,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:34,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:36,027 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 04:34:36,244 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 04:34:36,323 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 04:34:36,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 04:34:38,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:34:40,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:34:40,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:41,626 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:34:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:42,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:44,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:46,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:47,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:47,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:48,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:50,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:52,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:54,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:54,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:55,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:34:55,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:34:56,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:34:58,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:34:58,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:00,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:01,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:03,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:03,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:03,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:04,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:05,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:07,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:10,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:35:10,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:35:11,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:11,699 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:35:12,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:13,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:14,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:15,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:17,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:19,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:19,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:20,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:21,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:22,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:22,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:23,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:24,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:24,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:25,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:35:26,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:35:27,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:37,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:41,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:35:41,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:35:41,811 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:35:46,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:46,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:47,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:51,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:51,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:51,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:53,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:55,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:35:56,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:35:56,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:35:56,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:35:57,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:01,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:02,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:05,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:07,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:08,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:10,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:11,840 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:36:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:36:12,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:36:13,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:14,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:14,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:15,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:17,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:19,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:21,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:25,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:25,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:25,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:27,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:27,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:36:27,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:36:29,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:30,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:30,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:33,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:35,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:37,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:39,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:40,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:41,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:41,914 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:36:42,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:36:42,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:36:44,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:46,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:47,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:48,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:50,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:52,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:54,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:56,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:36:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:57,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:36:57,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:36:58,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:36:58,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:00,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:02,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:04,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:04,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:04,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:05,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:06,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:07,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:07,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:08,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:10,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:11,988 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:37:12,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:12,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:12,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:37:13,229 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:37:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:14,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:16,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:18,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:22,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:28,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:37:28,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:37:29,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:30,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:34,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:38,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:42,065 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:37:42,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:44,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:37:44,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:37:46,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:47,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:49,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:50,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:52,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:53,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:54,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:37:56,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:58,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:37:59,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:37:59,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:38:00,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:01,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:02,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:04,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:07,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:07,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:08,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:10,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:12,141 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:38:12,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:14,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:38:14,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:38:16,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:18,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:20,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:21,875 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 04:38:22,074 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 04:38:22,137 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 04:38:22,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 04:38:22,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:22,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:22,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:23,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:24,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:27,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:28,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:29,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:38:29,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:38:30,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:32,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:34,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:36,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:36,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:36,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:38,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:40,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:41,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:42,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:38:42,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:44,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:44,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:45,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:38:45,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:38:46,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:48,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:48,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:49,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:51,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:52,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:53,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:55,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:56,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:56,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:57,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:58,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:58,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:38:59,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:38:59,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:00,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:39:00,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:39:01,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:02,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:03,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:04,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:04,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:05,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:06,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:07,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:08,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:08,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:09,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:10,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:10,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:11,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:12,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:39:13,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:15,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:39:15,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:39:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:17,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:21,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:21,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:25,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:26,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:26,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:31,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:39:31,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:39:31,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:32,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:35,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:39,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:41,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:42,446 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:39:45,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:46,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:39:46,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:39:47,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:47,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:49,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:49,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:50,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:51,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:55,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:55,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:57,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:39:57,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:39:57,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:40:01,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:40:01,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:01,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:02,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:04,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:04,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:06,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:08,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:08,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:10,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:11,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:11,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:12,523 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:40:12,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:14,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:14,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:14,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:14,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:14,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:15,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:16,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:17,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:40:17,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:40:18,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:20,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:20,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:22,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:24,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:26,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:28,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:30,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:32,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:40:32,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:40:32,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:34,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:36,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:38,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:39,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:40,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:42,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:42,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:40:43,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:44,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:46,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:46,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:47,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:40:47,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:40:48,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:50,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:52,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:54,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:55,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:56,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:57,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:58,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:40:59,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:40:59,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:00,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:01,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:01,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:02,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:41:02,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:41:02,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:03,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:04,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:06,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:12,672 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:41:16,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:18,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:41:18,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:41:19,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:20,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:21,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:21,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:26,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:27,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:29,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:30,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:31,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:33,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:41:33,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:41:33,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:35,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:37,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:38,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:41,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:42,744 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:41:43,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:44,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:45,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:45,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:47,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:48,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:41:48,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:41:49,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:53,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:54,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:41:55,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:41:59,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:01,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:01,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:03,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:03,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:42:03,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:42:05,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:05,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:07,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:07,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:08,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 04:42:09,367 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 04:42:09,588 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 04:42:09,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:09,666 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 04:42:10,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 04:42:11,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:12,871 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:42:13,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:15,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:17,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:19,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:42:19,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:42:19,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:21,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:23,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:25,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:27,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:29,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:31,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:33,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:34,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:42:34,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:42:35,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:37,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:39,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:41,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:42,946 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:42:43,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:45,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:47,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:49,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:49,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:42:49,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:42:51,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:53,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:55,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:57,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:42:59,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:01,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:03,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:04,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:43:04,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:43:05,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:07,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:09,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:11,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:13,022 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:43:13,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:15,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:17,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:19,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:43:20,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:43:21,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:23,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:25,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:27,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:29,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:31,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:33,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:35,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:43:35,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:43:36,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:38,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:40,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:42,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:43,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:43:44,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:46,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:48,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:50,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:43:50,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:43:52,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:54,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:56,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:43:58,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:00,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:02,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:04,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:06,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:44:06,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:44:06,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:08,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:10,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:12,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:13,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:44:14,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:16,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:18,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:20,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:21,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:44:21,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:44:22,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:24,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:26,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:28,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:30,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:32,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:34,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:36,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:44:36,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:44:36,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:38,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:40,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:42,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:43,247 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:44:44,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:46,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:48,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:50,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:52,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:44:52,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:44:52,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:54,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:56,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:44:58,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:00,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:02,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:04,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:06,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:45:07,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:45:08,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:10,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:12,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:13,323 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:45:14,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:16,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:18,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:20,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:45:22,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:45:22,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:25,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:27,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:29,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:31,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:33,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:35,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:37,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:45:37,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:45:39,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:41,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:43,402 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:45:43,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:45,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:47,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:49,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:51,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:53,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:45:53,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:45:53,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:55,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:57,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:45:59,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:01,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:03,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:05,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:07,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:08,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:46:08,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:46:09,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:11,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:13,474 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:46:13,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:15,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:17,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:19,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:21,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:23,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:46:23,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:46:25,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:27,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:29,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:31,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:33,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:35,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:37,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:39,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:46:39,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:46:39,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:41,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:43,549 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:46:43,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:46,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:48,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:50,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:52,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:54,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:46:54,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:46:54,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:56,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:46:58,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:00,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:02,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:04,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:06,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:08,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:09,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:47:09,683 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:47:10,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:12,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:13,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:47:14,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:16,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:18,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:20,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:22,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:24,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:47:24,997 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:47:26,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:28,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:30,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:32,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:34,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:36,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:38,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:40,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:47:40,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:47:40,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:42,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:43,696 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:47:44,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:46,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:48,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:50,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:52,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:54,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:55,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:47:55,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:47:57,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:47:59,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:01,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:03,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:05,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:07,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:09,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:48:10,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:48:11,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:13,771 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:48:13,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:15,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:17,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:19,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:21,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:23,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:25,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:48:26,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:48:27,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:29,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:31,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:33,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:35,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:37,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:39,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:48:41,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:48:41,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:43,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:43,845 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:48:45,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:47,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:49,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:51,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:53,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:55,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:56,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:48:56,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:48:57,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:48:59,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:02,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:04,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:06,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:08,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:10,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:11,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:49:11,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:49:12,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:13,916 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:49:14,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:16,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:18,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:20,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:22,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:24,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:26,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:27,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:49:27,292 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:49:28,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:30,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:32,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:34,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:36,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:38,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:40,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:49:42,596 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:49:42,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:43,989 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:49:44,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:46,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:48,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:50,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:52,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:54,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:56,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:49:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:49:57,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:49:58,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:00,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:02,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:04,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:06,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:08,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:10,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:12,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:13,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:50:13,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:50:14,062 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:50:14,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:16,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:18,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:20,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:22,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:24,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:26,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:28,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:50:28,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:50:28,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:30,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:32,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:34,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:36,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:38,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:40,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:42,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:43,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:50:43,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:50:44,133 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:50:44,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:46,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:48,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:50,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:52,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:54,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:56,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:58,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:50:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:50:59,203 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:51:00,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:02,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:04,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:07,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:09,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:11,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:13,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:14,196 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:51:14,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:51:14,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:51:15,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:17,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:19,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:21,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:23,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:25,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:27,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:29,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:51:29,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:51:29,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:31,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:33,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:35,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:37,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:39,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:41,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:43,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:44,269 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:51:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:51:44,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:51:45,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:47,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:49,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:51,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:53,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:55,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:57,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:51:59,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:52:00,154 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:52:01,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:03,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:06,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:08,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:10,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:12,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:14,342 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:52:14,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:15,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:52:15,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:52:16,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:18,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:20,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:22,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:24,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:26,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:28,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:52:30,617 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:52:30,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:32,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:34,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:36,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:38,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:40,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:42,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:44,418 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:52:44,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:45,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:52:45,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:52:46,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:48,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:50,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:52,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:54,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:56,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:52:58,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:00,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:01,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:53:01,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:53:02,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:04,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:06,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:08,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:10,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:12,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:14,493 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:53:14,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:53:16,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:53:16,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:18,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:20,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:22,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:24,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:26,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:28,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:30,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:31,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:53:31,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:53:32,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:34,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:36,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:39,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:41,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:43,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:44,567 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:53:45,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:47,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:53:47,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:53:47,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:49,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:51,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:53,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:55,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:57,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:53:59,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:01,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:02,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:54:02,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:54:03,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:05,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:07,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:09,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:11,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:13,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:14,640 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:54:15,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:17,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:54:17,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:54:17,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:19,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:21,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:23,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:25,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:27,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:29,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:31,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:32,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:54:32,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:54:33,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:35,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:37,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:39,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:41,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:43,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:44,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:54:45,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:47,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:54:48,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:54:49,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:51,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:53,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:55,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:54:57,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:00,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:02,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:03,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:55:03,597 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:55:04,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:06,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:08,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:10,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:12,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:14,789 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:55:14,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:16,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:18,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:55:19,113 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:55:20,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:22,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:24,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:26,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:28,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:30,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:32,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:34,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:55:34,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:55:34,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:36,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:38,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:40,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:42,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:44,865 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:55:44,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:46,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:48,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:49,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:55:49,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:55:50,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:52,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:54,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:56,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:55:58,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:00,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:02,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:05,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:05,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:56:05,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:56:07,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:09,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:11,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:13,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:14,942 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:56:15,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:17,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:19,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:20,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:56:20,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:56:21,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:23,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:26,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:28,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:30,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:32,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:34,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:35,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:56:35,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:56:36,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:38,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:40,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:42,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:44,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:45,019 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:56:46,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:48,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:50,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:50,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:56:50,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:56:52,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:54,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:56,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:56:58,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:00,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:02,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:04,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:06,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:06,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:57:06,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:57:08,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:10,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:12,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:14,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:15,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:57:16,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:18,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:20,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:21,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:57:21,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:57:22,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:24,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:26,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:28,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:30,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:32,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:34,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:36,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:36,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:57:36,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:57:38,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:40,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:42,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:44,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:45,170 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:57:46,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:48,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:50,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:52,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:52,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:57:52,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:57:54,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:56,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:57:58,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:00,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:02,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:04,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:06,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:58:07,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:58:08,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:11,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:13,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:15,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:15,246 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:58:17,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:19,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:21,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:22,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:58:22,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:58:23,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:25,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:27,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:29,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:31,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:33,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:35,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:37,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:58:38,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:58:39,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:41,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:43,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:45,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:45,319 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:58:47,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:49,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:51,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:53,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:53,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:58:53,359 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:58:55,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:57,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:58:59,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:01,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:03,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:05,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:07,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:08,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:59:08,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:59:09,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:11,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:13,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:15,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:15,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:59:17,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:19,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:21,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:23,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:24,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:59:24,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:59:25,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:28,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:30,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:32,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:34,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:36,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:38,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:39,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:59:39,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:59:40,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:42,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:44,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:45,466 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 04:59:46,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:48,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:50,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:52,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:54,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:54,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 04:59:54,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 04:59:56,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 04:59:58,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:00,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:02,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:04,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:06,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:08,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:10,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:00:10,122 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:00:12,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:14,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:15,541 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:00:16,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:18,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:20,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:23,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:25,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:00:25,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:00:27,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:29,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:31,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:33,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:35,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:37,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:39,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:40,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:00:40,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:00:41,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:43,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:45,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:45,616 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:00:47,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:49,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:51,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:53,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:55,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:56,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:00:56,596 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:00:57,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:00:59,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:01,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:03,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:05,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:07,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:09,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:11,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:11,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:01:11,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:01:13,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:15,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:15,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:01:17,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:19,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:21,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:23,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:25,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:27,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:27,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:01:27,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:01:29,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:31,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:33,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:35,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:37,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:39,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:41,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:42,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:01:42,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:01:44,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:45,767 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:01:46,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:48,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:50,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:52,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:54,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:56,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:01:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:01:57,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:01:58,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:00,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:02,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:04,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:06,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:08,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:10,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:12,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:13,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:02:13,065 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:02:14,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:15,843 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:02:16,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:18,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:20,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:22,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:24,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:26,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:28,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:28,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:02:28,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:02:30,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:32,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:34,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:36,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:38,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:40,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:42,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:43,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:02:43,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:02:44,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:45,919 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:02:46,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:48,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:50,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:52,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:54,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:56,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:58,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:02:58,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:02:58,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:03:00,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:02,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:04,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:06,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:08,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:10,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:12,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:14,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:03:14,248 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:03:15,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:15,995 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:03:17,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:19,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:21,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:23,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:25,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:27,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:29,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:29,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:03:29,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:03:31,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:33,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:35,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:37,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:39,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:41,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:43,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:44,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:03:44,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:03:45,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:46,070 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:03:47,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:49,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:51,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:53,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:55,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:57,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:03:59,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:04:00,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:04:01,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:03,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:05,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:07,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:09,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:11,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:13,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:15,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:15,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:04:15,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:04:16,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:04:17,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:19,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:21,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:23,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:25,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:27,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:29,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:30,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:04:30,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:04:31,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:33,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:35,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:37,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:40,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:42,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:44,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:45,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:04:45,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:04:46,217 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:04:46,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:48,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:50,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:52,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:54,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:56,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:04:58,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:00,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:05:01,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:05:02,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:04,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:06,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:08,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:10,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:12,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:14,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:16,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:16,295 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:05:16,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:05:16,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:05:18,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:20,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:22,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:24,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:26,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:28,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:30,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:05:31,807 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:05:32,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:34,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:36,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:38,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:40,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:42,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:44,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:46,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:46,370 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:05:47,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:05:47,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:05:48,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:50,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:52,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:54,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:56,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:05:58,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:00,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:02,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:06:02,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:06:04,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:06,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:09,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:11,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:13,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:15,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:16,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:06:17,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:17,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:06:17,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:06:19,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:21,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:23,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:25,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:27,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:29,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:31,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:06:33,003 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:06:33,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:35,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:37,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:39,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:41,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:43,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:45,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:46,516 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:06:47,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:48,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:06:48,261 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:06:49,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:51,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:53,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:55,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:57,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:06:59,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:01,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:03,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:03,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:07:03,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:07:05,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:07,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:09,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:11,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:13,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:15,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:16,591 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:07:17,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:07:18,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:07:19,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:21,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:23,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:25,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:27,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:29,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:31,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:33,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:07:34,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:07:36,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:38,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:40,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:42,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:44,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:46,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:46,665 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:07:48,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:07:49,321 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:07:50,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:52,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:54,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:56,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:07:58,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:00,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:02,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:04,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:04,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:08:04,582 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:08:06,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:08,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:10,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:12,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:14,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:16,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:16,742 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:08:18,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:19,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:08:19,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:08:20,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:22,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:24,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:26,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:28,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:30,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:32,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:34,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:08:35,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:08:36,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:38,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:40,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:42,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:44,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:46,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:46,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:08:48,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:50,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:08:50,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:08:52,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:54,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:56,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:08:58,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:00,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:03,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:05,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:05,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:09:05,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:09:07,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:09,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:11,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:13,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:15,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:16,893 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:09:17,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:19,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:21,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:09:21,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:09:21,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:23,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:25,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:27,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:29,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:31,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:33,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:35,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:36,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:09:36,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:09:37,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:39,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:41,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:43,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:45,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:46,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:09:47,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:49,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:51,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:09:51,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:09:53,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:55,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:57,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:09:59,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:01,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:03,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:05,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:10:06,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:10:07,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:09,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:11,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:13,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:15,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:17,041 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:10:17,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:19,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:21,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:22,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:10:22,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:10:23,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:25,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:28,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:30,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:32,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:34,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:36,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:37,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:10:37,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:10:38,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:40,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:42,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:44,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:46,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:47,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:10:48,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:50,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:52,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:52,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:10:52,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:10:54,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:56,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:10:58,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:00,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:02,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:04,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:06,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:11:08,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:11:08,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:10,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:12,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:14,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:16,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:17,190 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:11:18,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:20,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:22,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:11:23,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:11:24,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:26,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:28,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:30,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:32,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:34,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:36,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:38,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:38,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:11:38,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:11:40,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:42,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:45,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:47,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:11:47,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:49,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:51,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:53,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:53,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:11:53,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:11:55,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:57,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:11:59,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:01,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:03,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:05,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:07,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:09,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:10,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:12:10,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:12:11,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:13,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:15,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:17,336 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:12:17,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:19,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:21,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:23,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:25,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:25,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:12:25,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:12:27,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:29,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:31,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:33,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:35,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:37,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:39,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:41,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:12:41,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:12:41,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:43,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:45,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:47,411 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:12:47,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:49,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:51,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:53,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:55,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:56,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:12:56,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:12:57,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:12:59,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:01,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:03,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:05,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:07,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:09,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:11,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:11,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:13:11,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:13:13,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:15,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:17,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:17,484 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:13:19,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:21,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:23,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:25,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:26,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:13:26,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:13:27,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:30,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:32,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:34,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:36,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:38,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:40,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:42,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:13:42,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:13:42,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:44,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:46,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:47,561 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:13:48,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:50,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:52,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:54,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:56,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:13:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:13:57,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:13:58,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:00,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:02,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:04,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:06,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:08,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:10,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:12,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:14:12,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:14:14,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:16,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:17,637 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:14:18,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:20,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:22,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:24,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:26,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:28,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:14:28,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:14:28,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:30,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:32,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:34,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:36,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:38,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:40,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:42,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:43,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:14:43,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:14:45,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:47,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:47,715 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:14:49,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:51,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:53,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:55,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:57,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:14:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:14:58,637 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:14:59,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:01,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:03,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:05,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:07,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:09,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:11,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:13,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:13,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:15:13,945 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:15:15,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:17,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:17,791 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:15:19,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:21,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:23,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:25,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:27,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:15:29,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:15:29,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:31,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:33,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:35,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:37,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:39,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:41,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:43,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:44,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:15:44,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:15:45,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:47,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:47,866 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:15:49,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:51,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:53,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:55,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:57,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:59,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:15:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:15:59,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:16:01,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:03,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:05,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:07,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:09,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:11,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:13,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:15,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:16:15,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:16:15,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:17,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:17,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:16:19,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:21,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:23,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:25,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:27,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:29,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:30,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:16:30,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:16:31,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:34,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:36,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:38,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:40,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:42,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:44,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:45,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:16:45,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:16:46,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:48,019 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:16:48,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:50,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:52,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:54,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:56,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:16:58,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:00,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:00,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:17:00,795 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:17:02,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:04,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:06,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:08,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:10,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:12,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:14,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:16,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:17:16,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:17:16,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:18,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:17:18,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:20,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:22,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:24,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:26,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:28,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:30,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:31,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:17:31,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:17:32,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:34,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:36,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:38,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:40,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:42,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:44,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:46,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:17:46,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:17:46,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:48,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:17:48,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:50,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:52,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:55,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:57,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:17:59,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:01,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:18:01,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:18:03,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:05,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:07,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:09,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:11,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:13,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:15,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:18:17,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:18:17,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:18,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:18:19,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:21,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:23,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:25,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:27,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:29,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:31,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:18:32,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:18:33,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:35,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:37,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:39,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:41,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:43,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:45,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:47,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:47,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:18:47,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:18:48,326 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:18:49,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:51,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:53,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:55,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:57,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:18:59,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:01,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:02,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:19:02,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:19:03,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:05,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:07,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:09,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:11,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:13,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:15,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:17,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:19:18,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:19:18,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:19:20,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:22,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:24,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:26,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:28,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:30,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:32,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:19:33,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:19:34,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:36,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:38,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:40,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:42,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:44,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:46,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:48,478 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:19:48,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:48,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:19:48,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:19:50,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:52,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:54,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:56,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:19:58,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:00,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:02,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:04,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:20:04,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:20:04,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:06,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:08,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:10,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:12,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:14,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:16,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:18,555 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:20:18,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:19,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:20:19,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:20:20,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:22,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:24,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:26,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:28,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:30,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:32,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:34,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:20:34,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:20:36,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:38,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:40,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:42,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:45,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:47,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:48,630 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:20:49,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:20:50,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:20:51,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:53,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:55,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:57,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:20:59,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:01,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:03,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:05,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:21:05,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:21:05,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:07,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:09,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:11,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:13,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:15,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:17,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:18,706 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:21:19,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:21:20,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:21:21,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:23,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:25,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:27,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:29,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:31,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:33,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:35,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:35,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:21:35,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:21:37,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:39,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:41,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:43,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:45,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:47,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:48,799 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:21:49,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:51,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:21:51,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:21:51,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:53,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:55,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:57,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:21:59,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:01,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:03,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:05,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:06,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:22:06,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:22:07,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:09,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:11,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:13,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:16,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:18,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:18,877 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:22:20,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:21,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:22:21,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:22:22,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:24,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:26,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:28,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:30,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:32,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:34,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:36,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:22:37,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:22:38,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:40,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:42,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:44,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:46,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:48,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:48,949 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:22:50,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:52,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:22:52,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:22:52,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:54,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:56,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:22:58,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:00,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:02,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:04,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:06,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:07,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:23:07,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:23:08,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:10,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:12,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:14,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:16,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:18,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:19,010 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:23:20,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:22,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:22,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:23:22,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:23:24,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:26,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:29,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:31,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:33,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:35,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:37,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:23:38,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:23:39,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:41,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:43,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:45,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:47,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:49,073 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:23:49,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:51,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:53,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:23:53,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:23:53,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:55,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:57,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:23:59,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:01,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:03,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:05,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:07,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:08,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:24:08,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:24:09,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:11,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:13,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:15,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:17,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:19,136 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:24:19,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:21,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:23,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:23,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:24:23,863 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:24:25,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:27,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:29,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:31,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:33,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:35,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:37,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:39,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:24:39,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:24:39,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:24:49,200 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:24:54,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:24:54,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:25:09,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:25:09,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:25:19,267 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:25:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:25:24,899 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:25:40,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:25:40,227 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:25:41,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:25:41,006 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 05:25:41,056 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 05:25:41,134 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 05:25:41,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 05:25:41,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:25:42,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:25:47,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:25:49,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:25:55,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:25:55,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:26:03,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:26:10,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:26:10,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:26:19,436 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:26:25,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:26:25,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:26:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:26:41,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:26:49,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:26:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:26:56,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:27:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:27:11,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:27:19,592 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:27:27,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:27:27,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:27:42,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:27:42,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:27:49,669 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:27:57,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:27:57,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:28:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:28:12,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:28:19,745 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:28:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:28:28,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:28:43,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:28:43,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:28:49,817 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:28:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:28:58,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:29:13,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:29:13,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:29:19,887 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:29:28,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:29:28,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:29:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:29:44,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:29:49,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:29:56,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:29:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:29:59,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:30:00,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:02,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:04,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:05,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:06,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:07,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:08,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:10,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:10,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:12,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:30:14,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:30:14,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:16,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:18,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:18,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:20,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:30:20,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:21,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:22,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:24,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:26,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:26,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:28,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:29,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:29,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:30:29,971 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:30:30,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:31,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:31,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:32,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:34,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:35,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:35,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:36,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:37,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:38,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:39,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:39,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:40,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:41,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:41,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:42,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:43,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:43,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:44,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:45,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:30:45,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:30:46,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:50,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:30:54,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:30:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:30:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:31:00,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:31:00,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:00,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:01,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:01,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:04,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:06,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:06,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:08,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:09,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:09,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:10,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:11,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:13,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:14,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:31:15,801 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:31:18,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:19,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:19,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:20,168 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:31:23,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:25,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:26,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:27,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:29,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:31,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:31:31,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:31:31,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:35,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:37,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:39,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:43,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:45,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:46,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:46,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:31:46,411 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:31:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:47,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:50,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:31:51,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:52,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:52,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:53,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:55,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:31:56,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:56,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:31:57,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:01,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:32:01,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:32:01,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:02,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:02,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:03,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:05,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:05,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:06,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:06,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:07,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:09,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:11,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:13,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:15,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:16,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:32:16,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:32:17,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:19,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:20,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:32:21,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:21,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:23,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:25,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:26,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:26,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:27,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:29,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:30,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:30,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:31,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:32,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:32:32,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:32:33,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:34,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:34,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:35,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:36,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:36,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:37,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:38,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:39,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:40,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:40,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:41,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:43,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:47,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:32:47,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:32:49,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:49,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:50,364 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:32:51,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:52,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:52,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:57,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:32:59,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:32:59,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:01,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:02,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:33:02,624 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:33:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:04,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:05,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:09,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:13,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:14,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:14,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:17,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:33:17,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:33:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:19,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:20,437 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:33:21,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:22,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:23,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:24,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:24,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:27,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:29,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:31,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:33:33,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:33:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:33,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:35,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:37,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:41,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:41,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:43,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:45,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:45,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:46,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:47,173 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 05:33:47,390 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 05:33:47,467 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 05:33:47,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 05:33:47,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:33:48,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:33:48,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:49,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:50,573 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:33:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:51,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:53,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:55,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:56,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:56,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:33:57,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:33:59,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:01,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:01,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:03,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:34:03,795 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:34:04,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:04,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:06,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:06,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:08,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:10,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:12,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:14,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:16,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:16,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:18,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:19,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:34:19,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:34:20,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:20,646 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:34:20,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:22,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:24,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:25,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:26,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:27,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:28,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:29,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:29,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:30,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:31,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:32,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:34,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:34:34,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:34:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:34,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:35,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:36,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:38,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:40,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:41,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:45,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:48,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:49,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:34:49,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:34:50,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:34:52,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:53,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:56,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:58,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:34:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:34:59,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:02,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:35:05,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:35:06,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:06,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:07,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:07,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:08,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:09,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:10,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:12,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:13,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:16,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:35:20,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:35:20,790 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:35:20,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:21,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:22,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:23,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:23,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:26,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:29,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:31,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:33,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:35,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:35,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:35:35,754 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:35:37,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:41,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:43,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:43,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:43,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:46,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:47,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:47,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:48,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:50,867 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:35:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:35:50,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:35:51,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:51,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:52,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:53,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:35:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:56,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:35:58,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:00,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:01,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:01,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:02,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:04,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:05,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:05,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:06,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:36:06,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:36:07,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:08,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:08,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:10,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:12,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:14,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:16,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:16,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:18,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:20,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:20,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:20,940 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:36:21,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:36:21,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:36:22,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:23,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:24,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:25,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:26,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:26,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:28,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:28,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:30,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:32,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:32,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:32,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:34,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:36,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:36,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:36:36,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:36:46,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:48,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:48,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:50,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:51,012 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:36:52,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:36:52,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:36:56,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:36:57,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:36:58,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:00,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:04,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:05,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:05,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:06,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:37:07,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:37:07,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:07,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:09,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:10,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:14,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:14,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:16,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:20,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:20,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:21,082 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:37:22,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:22,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:37:22,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:37:23,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:23,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:25,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:26,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:27,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:28,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:29,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:29,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:30,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:31,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:31,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:34,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:35,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:36,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:37:37,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:37:38,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:38,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:39,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:39,233 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 05:37:39,401 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 05:37:39,469 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 05:37:40,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 05:37:40,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:41,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:42,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:44,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:46,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:48,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:49,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:50,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:51,198 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:37:52,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:52,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:37:53,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:37:53,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:53,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:54,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:55,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:55,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:37:56,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:37:59,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:00,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:00,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:01,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:03,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:04,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:04,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:05,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:07,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:08,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:38:08,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:38:08,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:08,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:09,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:09,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:11,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:13,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:14,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:14,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:15,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:17,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:19,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:20,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:20,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:21,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:21,276 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:38:23,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:23,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:38:23,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:38:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:25,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:26,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:27,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:27,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:29,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:31,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:38,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:38,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:38:38,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:38:40,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:41,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:44,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:45,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:49,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:51,350 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:38:52,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:52,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:53,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:54,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:38:54,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:38:56,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:56,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:57,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:38:58,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:38:58,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:01,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:05,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:07,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:07,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:07,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:39:09,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:39:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:11,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:13,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:16,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:17,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:19,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:19,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:19,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:20,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:21,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:21,425 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:39:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:23,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:39:24,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:39:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:24,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:25,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:27,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:27,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:29,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:30,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:31,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:32,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:35,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:37,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:39,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:39,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:39:39,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:39:41,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:41,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:41,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:43,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:43,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:45,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:46,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:47,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:49,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:49,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:49,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:51,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:39:51,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:51,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:53,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:54,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:39:55,156 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:39:55,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:55,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:57,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:59,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:39:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:39:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:00,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:01,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:03,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:03,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:03,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:05,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:07,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:09,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:10,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:40:10,392 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:40:11,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:13,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:14,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:15,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:15,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:17,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:18,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:19,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:20,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:21,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:21,572 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:40:23,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:25,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:40:25,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:40:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:31,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:34,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:36,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:38,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:41,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:40:41,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:40:42,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:43,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:46,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:50,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:51,648 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:40:52,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:52,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:54,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:40:56,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:40:58,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:40:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:40:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:00,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:04,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:05,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:05,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:06,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:07,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:07,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:07,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:10,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:41:11,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:41:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:11,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:12,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:14,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:15,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:16,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:17,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:18,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:19,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:20,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:21,725 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:41:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:22,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:23,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:24,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:25,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:26,216 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 05:41:26,427 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 05:41:26,490 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 05:41:26,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:41:26,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:41:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:27,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 05:41:28,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:30,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:30,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:32,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:34,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:38,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:40,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:41,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:41:41,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:41:42,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:44,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:45,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:46,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:48,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:50,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:51,809 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:41:52,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:54,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:56,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:56,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:56,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:41:57,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:41:58,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:41:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:41:58,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:01,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:02,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:04,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:06,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:08,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:10,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:11,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:12,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:12,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:42:12,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:42:13,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:13,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:14,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:15,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:15,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:16,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:18,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:21,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:42:26,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:27,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:42:27,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:42:30,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:30,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:34,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:36,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:38,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:40,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:42,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:42:42,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:42:46,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:50,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:51,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:42:52,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:55,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:56,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:42:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:57,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:42:58,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:42:58,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:42:58,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:02,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:02,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:04,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:05,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:06,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:08,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:10,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:12,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:13,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:43:13,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:43:15,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:16,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:17,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:18,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:19,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:21,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:22,024 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:43:22,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:22,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:23,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:24,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:25,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:43:28,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:43:29,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:31,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:33,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:35,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:35,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:35,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:36,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:37,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:39,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:40,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:40,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:41,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:41,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:43,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:44,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:43:44,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:43:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:45,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:47,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:47,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:49,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:51,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:51,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:51,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:52,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:43:53,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:54,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:54,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:55,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:57,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:58,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:58,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:43:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:43:59,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:43:59,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:44:01,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:03,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:05,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:07,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:08,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:08,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:09,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:11,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:14,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:44:14,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:44:21,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:21,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:21,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:22,171 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:44:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:25,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:28,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:28,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:29,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:44:29,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:44:31,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:33,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:37,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:39,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:41,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:42,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:43,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:44,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:44:45,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:44:46,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:48,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:52,247 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:44:52,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:54,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:58,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:44:59,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:44:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:00,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:45:00,629 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:45:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:00,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:04,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:06,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:07,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:08,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:10,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:11,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:13,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:14,377 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 05:45:14,590 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 05:45:14,679 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 05:45:15,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:15,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 05:45:15,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:45:15,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:45:16,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:17,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:17,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:18,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:20,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:20,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:20,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:22,627 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:45:22,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:22,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:24,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:26,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:28,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:28,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:28,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:30,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:31,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:45:31,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:45:32,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:34,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:34,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:35,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:36,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:38,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:40,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:41,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:42,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:44,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:44,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:44,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:46,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:46,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:45:46,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:45:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:47,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:48,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:50,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:52,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:52,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:52,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:45:53,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:54,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:55,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:55,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:45:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:45:58,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:00,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:01,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:46:01,794 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:46:02,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:04,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:14,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:14,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:14,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:46:17,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:46:17,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:17,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:19,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:21,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:22,778 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:46:23,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:24,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:24,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:29,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:31,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:32,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:32,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:46:32,392 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:46:35,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:36,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:39,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:40,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:43,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:45,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:46,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:46,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:47,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:46:47,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:46:49,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:51,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:52,851 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:46:55,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:57,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:46:59,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:46:59,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:01,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:02,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:47:02,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:47:03,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:03,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:05,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:06,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:06,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:07,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:07,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:08,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:09,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:09,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:10,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:11,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:13,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:15,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:16,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:17,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:47:18,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:47:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:19,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:20,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:21,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:22,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:22,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:47:23,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:25,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:26,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:26,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:27,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:29,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:31,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:31,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:31,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:33,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:33,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:47:33,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:47:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:35,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:36,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:36,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:38,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:39,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:40,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:44,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:46,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:46,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:48,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:48,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:48,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:47:48,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:47:50,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:50,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:50,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:52,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:52,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:47:53,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:53,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:47:54,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:56,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:47:58,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:00,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:48:04,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:48:07,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:07,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:08,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:12,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:18,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:19,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:48:19,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:48:20,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:20,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:23,083 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:48:24,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:27,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:28,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:29,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:29,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:32,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:34,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:34,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:48:34,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:48:35,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:36,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:37,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:38,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:40,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:44,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:44,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:44,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:46,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:49,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:48:49,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:48:50,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:52,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:53,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:48:54,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:56,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:48:58,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:59,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:59,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:59,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:48:59,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 05:49:00,527 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 05:49:00,732 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 05:49:00,793 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 05:49:01,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 05:49:02,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:04,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:49:05,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:49:06,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:08,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:10,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:12,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:14,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:16,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:18,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:20,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:20,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:49:20,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:49:22,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:23,265 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:49:24,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:26,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:28,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:30,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:32,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:34,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:35,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:49:35,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:49:36,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:38,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:40,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:42,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:45,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:47,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:49,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:49:51,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:49:51,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:53,339 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:49:53,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:55,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:57,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:49:59,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:01,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:03,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:05,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:50:06,282 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:50:07,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:09,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:11,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:13,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:15,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:17,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:19,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:21,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:21,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:50:21,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:50:23,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:50:23,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:25,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:27,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:29,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:31,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:33,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:35,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:36,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:50:36,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:50:37,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:39,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:41,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:43,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:45,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:47,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:49,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:51,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:52,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:50:52,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:50:53,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:53,480 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:50:55,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:57,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:50:59,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:01,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:03,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:05,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:07,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:07,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:51:07,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:51:09,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:12,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:14,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:16,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:18,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:20,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:22,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:51:22,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:51:23,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:51:24,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:26,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:28,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:30,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:32,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:34,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:36,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:37,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:51:37,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:51:38,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:40,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:42,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:44,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:46,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:48,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:50,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:52,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:53,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:51:53,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:51:53,636 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:51:54,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:56,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:51:58,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:00,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:02,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:04,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:06,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:08,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:08,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:52:08,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:52:10,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:12,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:14,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:16,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:18,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:20,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:22,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:23,708 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:52:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:52:23,788 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:52:24,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:27,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:29,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:31,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:33,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:35,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:37,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:39,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:52:39,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:52:39,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:41,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:43,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:45,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:47,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:49,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:51,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:53,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:53,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:52:54,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:52:54,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:52:55,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:57,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:52:59,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:01,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:03,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:05,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:07,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:09,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:09,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:53:09,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:53:11,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:13,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:15,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:17,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:19,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:21,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:23,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:23,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:53:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:53:24,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:53:25,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:28,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:30,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:32,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:34,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:36,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:38,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:40,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:53:40,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:53:40,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:42,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:44,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:46,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:48,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:50,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:52,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:53,935 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:53:54,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:55,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:53:55,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:53:56,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:53:58,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:00,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:02,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:04,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:06,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:08,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:10,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:54:10,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:54:12,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:14,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:16,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:18,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:20,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:22,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:24,009 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:54:24,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:25,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:54:25,953 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:54:26,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:28,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:30,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:32,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:34,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:36,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:38,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:40,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:41,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:54:41,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:54:42,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:44,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:46,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:48,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:51,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:53,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:54,086 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:54:55,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:56,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:54:56,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:54:57,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:54:59,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:01,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:03,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:05,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:07,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:09,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:11,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:55:11,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:55:13,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:15,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:17,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:19,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:21,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:23,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:24,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:55:25,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:27,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:55:27,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:55:27,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:29,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:31,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:33,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:35,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:37,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:39,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:41,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:55:42,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:55:43,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:45,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:47,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:49,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:51,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:53,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:54,238 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:55:55,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:57,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:55:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:55:57,742 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:55:59,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:01,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:03,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:05,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:07,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:09,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:11,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:56:12,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:56:13,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:15,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:17,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:19,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:21,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:23,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:24,316 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:56:25,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:27,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:28,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:56:28,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:56:29,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:32,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:34,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:36,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:38,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:40,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:42,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:43,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:56:43,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:56:44,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:46,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:48,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:50,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:52,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:54,390 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:56:54,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:56,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:58,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:56:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:56:58,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:57:00,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:02,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:04,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:06,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:08,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:10,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:12,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:14,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:57:14,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:57:14,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:16,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:18,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:20,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:22,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:24,471 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:57:24,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:26,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:28,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:29,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:57:29,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:57:30,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:32,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:34,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:36,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:38,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:40,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:42,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:44,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:57:44,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:57:44,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:46,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:49,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:51,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:53,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:54,546 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:57:55,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:57,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:59,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:57:59,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:57:59,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:58:01,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:03,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:05,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:07,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:09,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:11,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:13,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:15,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:58:15,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:58:15,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:17,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:19,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:21,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:23,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:24,617 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:58:25,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:27,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:29,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:58:30,373 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:58:31,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:33,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:35,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:37,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:39,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:41,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:43,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:45,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:58:45,602 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:58:45,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:47,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:49,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:51,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:53,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:54,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:58:55,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:57,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:58:59,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:00,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:59:00,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:59:01,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:03,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:05,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:07,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:09,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:11,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:13,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:15,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:16,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:59:16,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:59:17,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:19,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:21,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:23,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:24,764 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:59:25,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:27,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:29,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:31,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:59:31,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:59:31,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:33,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:36,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:38,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:40,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:42,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:44,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:46,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:46,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 05:59:46,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 05:59:48,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:50,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:52,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:54,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:54,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 05:59:56,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 05:59:58,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:00,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:00:02,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:00:02,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:04,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:06,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:08,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:10,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:12,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:14,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:16,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:00:17,411 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:00:18,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:20,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:22,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:24,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:24,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:00:26,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:28,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:30,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:32,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:00:32,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:00:32,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:34,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:36,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:38,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:40,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:42,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:44,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:47,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:47,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:00:47,894 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:00:49,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:51,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:53,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:54,981 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:00:55,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:57,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:00:59,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:01,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:03,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:01:03,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:01:03,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:05,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:07,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:09,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:11,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:13,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:15,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:17,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:01:18,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:01:19,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:21,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:23,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:25,051 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:01:25,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:27,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:29,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:31,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:33,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:01:33,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:01:33,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:35,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:37,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:39,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:41,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:43,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:45,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:47,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:48,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:01:48,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:01:49,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:51,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:54,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:55,126 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:01:56,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:01:58,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:00,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:02,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:04,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:02:04,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:02:04,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:06,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:08,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:10,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:12,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:14,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:16,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:18,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:19,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:02:19,491 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:02:20,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:22,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:24,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:25,203 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:02:26,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:28,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:30,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:32,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:34,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:02:34,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:02:36,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:38,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:40,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:42,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:44,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:46,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:48,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:02:50,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:02:50,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:52,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:54,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:55,278 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:02:56,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:02:58,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:00,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:02,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:04,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:05,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:03:05,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:03:06,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:09,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:11,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:13,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:15,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:17,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:19,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:20,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:03:20,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:03:21,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:23,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:25,354 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:03:25,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:27,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:29,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:31,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:33,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:35,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:35,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:03:35,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:03:37,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:39,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:41,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:43,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:45,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:47,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:49,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:03:51,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:03:51,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:53,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:55,422 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:03:55,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:57,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:03:59,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:01,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:03,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:05,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:06,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:04:06,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:04:07,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:09,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:11,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:13,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:15,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:17,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:19,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:21,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:04:21,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:04:21,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:23,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:25,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:04:25,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:27,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:29,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:31,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:33,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:35,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:04:36,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:04:37,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:39,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:41,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:43,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:45,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:47,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:49,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:51,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:04:52,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:04:54,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:55,557 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:04:56,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:04:58,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:00,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:02,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:04,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:06,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:05:07,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:05:08,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:10,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:12,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:14,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:16,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:18,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:20,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:22,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:05:23,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:05:24,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:25,627 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:05:26,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:28,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:30,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:32,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:34,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:36,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:05:38,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:05:38,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:40,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:42,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:44,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:46,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:48,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:50,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:52,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:53,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:05:53,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:05:54,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:55,702 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:05:56,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:05:58,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:00,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:02,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:04,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:06,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:08,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:06:08,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:06:08,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:10,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:12,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:14,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:16,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:18,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:21,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:23,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:24,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:06:24,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:06:25,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:06:25,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:27,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:29,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:31,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:33,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:35,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:37,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:39,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:06:39,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:06:39,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:41,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:43,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:45,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:47,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:49,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:51,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:53,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:54,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:06:54,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:06:55,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:55,855 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:06:57,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:06:59,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:01,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:03,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:05,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:07,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:09,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:07:09,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:07:11,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:13,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:15,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:17,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:19,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:21,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:23,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:25,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:07:25,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:07:25,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:25,930 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:07:27,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:29,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:31,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:33,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:35,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:37,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:39,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:40,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:07:40,530 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:07:41,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:43,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:45,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:47,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:49,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:51,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:53,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:07:55,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:07:56,029 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:07:56,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:07:58,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:00,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:02,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:04,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:06,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:08,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:10,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:11,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:08:11,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:08:12,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:14,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:16,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:18,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:20,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:22,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:24,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:26,080 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:08:26,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:08:26,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:08:26,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:28,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:30,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:32,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:34,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:36,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:38,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:40,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:41,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:08:41,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:08:42,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:44,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:46,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:48,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:50,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:52,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:54,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:56,157 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:08:56,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:08:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:08:56,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:08:59,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:01,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:03,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:05,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:07,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:09,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:11,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:09:12,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:09:13,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:15,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:17,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:19,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:21,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:23,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:25,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:26,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:09:27,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:09:27,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:09:27,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:29,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:31,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:33,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:35,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:37,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:39,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:41,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:42,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:09:42,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:09:43,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:45,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:47,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:49,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:51,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:53,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:55,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:56,312 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:09:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:09:57,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:09:57,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:09:59,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:01,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:03,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:05,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:07,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:09,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:11,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:13,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:10:13,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:10:13,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:15,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:17,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:19,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:21,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:23,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:25,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:26,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:10:27,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:28,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:10:28,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:10:29,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:31,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:33,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:35,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:37,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:39,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:42,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:43,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:10:43,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:10:44,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:46,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:48,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:50,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:52,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:54,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:56,469 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:10:56,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:10:58,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:10:58,944 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:10:58,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:00,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:02,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:04,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:06,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:08,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:10,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:12,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:11:14,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:11:14,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:16,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:18,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:20,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:22,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:24,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:26,542 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:11:26,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:28,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:29,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:11:29,414 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:11:30,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:32,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:34,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:36,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:38,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:40,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:42,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:44,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:11:44,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:11:44,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:46,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:49,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:51,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:53,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:55,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:56,620 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:11:57,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:11:59,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:11:59,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:11:59,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:01,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:03,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:05,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:07,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:09,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:11,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:13,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:15,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:12:15,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:12:15,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:17,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:19,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:21,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:23,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:25,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:26,696 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:12:27,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:29,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:30,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:12:30,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:12:31,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:33,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:35,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:37,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:39,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:42,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:44,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:45,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:12:45,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:12:46,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:48,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:50,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:52,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:54,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:56,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:12:56,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:12:58,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:01,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:13:01,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:13:03,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:05,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:07,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:09,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:11,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:13,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:15,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:13:16,268 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:13:17,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:19,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:21,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:23,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:25,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:26,849 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:13:27,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:29,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:31,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:31,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:13:31,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:13:33,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:35,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:37,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:39,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:41,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:43,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:45,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:46,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:13:46,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:13:47,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:49,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:51,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:53,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:55,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:56,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:13:57,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:13:59,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:01,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:02,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:14:02,082 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:14:03,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:05,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:07,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:10,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:12,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:14,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:16,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:17,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:14:17,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:14:18,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:20,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:22,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:24,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:26,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:26,996 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:14:28,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:30,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:32,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:32,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:14:32,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:14:34,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:36,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:38,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:40,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:42,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:44,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:46,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:47,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:14:47,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:14:48,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:50,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:52,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:54,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:56,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:14:57,079 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:14:58,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:00,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:02,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:03,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:15:03,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:15:04,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:06,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:08,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:10,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:12,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:14,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:16,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:18,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:18,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:15:18,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:15:20,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:22,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:24,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:26,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:27,153 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:15:28,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:30,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:32,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:15:33,754 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:15:34,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:36,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:38,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:40,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:42,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:44,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:46,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:48,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:49,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:15:49,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:15:50,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:52,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:55,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:57,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:15:57,229 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:15:59,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:01,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:03,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:04,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:16:04,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:16:05,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:07,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:09,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:11,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:13,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:15,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:17,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:19,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:16:19,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:16:21,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:23,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:25,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:27,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:27,306 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:16:29,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:31,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:33,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:34,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:16:34,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:16:35,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:37,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:39,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:41,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:43,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:45,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:47,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:49,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:50,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:16:50,106 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:16:51,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:53,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:55,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:57,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:16:57,384 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:16:59,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:01,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:03,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:05,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:05,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:17:05,371 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:17:07,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:10,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:12,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:14,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:16,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:18,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:20,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:17:20,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:17:22,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:24,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:26,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:27,461 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:17:28,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:30,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:32,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:34,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:35,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:17:35,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:17:36,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:38,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:40,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:42,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:44,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:46,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:48,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:50,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:51,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:17:51,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:17:52,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:54,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:56,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:17:57,537 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:17:58,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:00,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:02,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:04,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:06,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:18:06,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:18:08,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:10,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:12,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:14,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:16,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:18,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:20,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:21,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:18:21,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:18:22,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:24,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:26,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:27,610 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:18:28,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:30,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:32,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:34,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:36,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:36,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:18:36,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:18:38,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:40,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:42,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:44,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:46,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:48,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:50,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:52,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:52,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:18:52,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:18:54,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:56,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:18:57,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:18:58,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:00,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:02,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:04,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:06,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:07,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:19:07,626 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:19:08,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:10,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:13,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:15,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:17,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:19,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:21,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:19:22,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:19:23,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:25,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:27,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:27,765 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:19:29,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:31,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:33,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:35,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:37,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:38,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:19:38,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:19:39,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:41,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:43,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:45,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:47,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:49,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:51,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:53,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:53,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:19:53,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:19:55,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:57,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:19:57,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:19:59,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:01,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:03,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:05,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:07,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:08,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:20:08,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:20:09,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:11,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:13,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:16,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:18,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:20,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:22,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:24,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:20:24,154 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:20:24,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:26,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:27,923 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:20:28,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:30,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:32,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:34,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:36,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:38,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:39,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:20:39,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:20:40,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:42,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:44,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:46,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:48,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:50,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:52,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:54,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:20:54,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:20:56,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:20:57,999 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:20:58,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:00,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:02,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:04,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:06,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:08,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:09,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:21:09,867 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:21:10,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:12,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:14,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:16,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:18,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:20,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:22,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:24,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:25,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:21:25,094 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:21:26,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:28,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:21:28,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:30,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:32,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:34,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:36,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:38,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:40,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:40,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:21:40,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:21:42,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:44,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:46,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:48,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:50,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:52,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:54,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:55,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:21:55,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:21:56,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:21:58,149 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:21:58,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:00,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:02,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:04,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:07,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:09,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:10,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:22:10,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:22:11,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:13,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:15,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:17,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:19,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:21,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:23,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:25,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:22:26,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:22:27,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:28,222 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:22:29,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:31,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:33,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:35,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:37,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:39,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:41,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:41,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:22:41,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:22:43,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:45,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:47,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:49,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:51,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:53,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:55,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:22:56,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:22:57,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:22:58,298 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:22:59,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:01,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:03,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:05,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:07,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:09,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:11,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:12,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:23:12,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:23:13,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:15,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:17,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:19,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:21,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:23,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:25,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:27,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:23:27,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:23:28,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:23:29,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:31,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:33,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:35,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:38,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:40,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:42,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:42,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:23:42,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:23:44,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:46,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:48,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:50,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:52,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:54,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:56,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:58,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:23:58,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:23:58,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:23:58,448 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:24:00,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:02,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:04,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:06,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:08,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:10,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:12,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:24:13,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:24:14,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:16,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:18,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:20,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:22,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:24,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:26,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:28,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:28,525 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:24:28,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:24:28,731 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:24:30,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:32,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:34,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:36,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:38,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:40,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:42,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:43,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:24:43,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:24:44,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:46,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:48,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:50,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:52,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:54,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:56,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:58,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:24:58,603 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:24:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:24:59,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:25:00,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:02,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:04,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:06,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:08,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:10,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:12,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:14,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:14,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:25:14,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:25:16,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:18,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:20,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:22,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:25,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:27,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:28,687 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:25:29,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:29,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:25:29,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:25:31,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:33,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:35,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:37,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:39,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:41,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:43,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:45,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:25:45,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:25:45,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:47,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:49,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:51,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:53,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:55,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:57,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:25:58,765 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:25:59,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:26:00,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:26:01,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:03,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:05,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:07,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:09,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:11,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:13,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:15,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:15,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:26:15,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:26:17,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:19,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:21,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:23,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:25,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:27,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:28,841 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:26:29,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:31,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:26:31,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:26:31,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:33,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:35,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:37,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:40,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:42,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:44,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:46,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:46,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:26:46,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:26:48,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:50,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:52,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:54,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:56,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:58,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:26:58,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:27:00,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:01,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:27:01,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:27:02,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:04,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:06,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:08,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:10,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:12,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:14,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:16,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:16,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:27:16,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:27:18,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:20,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:22,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:24,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:26,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:28,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:28,998 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:27:30,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:32,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:27:32,273 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:27:32,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:34,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:36,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:38,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:40,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:42,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:44,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:46,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:47,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:27:47,516 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:27:48,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:50,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:52,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:54,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:56,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:58,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:27:59,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:28:00,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:02,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:28:02,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:28:04,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:06,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:08,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:10,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:12,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:14,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:17,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:18,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:28:18,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:28:19,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:21,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:23,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:25,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:27,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:29,150 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:28:29,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:31,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:28:33,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:28:33,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:35,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:37,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:39,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:41,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:43,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:45,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:47,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:48,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:28:48,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:28:49,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:51,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:53,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:55,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:57,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:28:59,224 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:28:59,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:01,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:03,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:03,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:29:03,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:29:05,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:07,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:09,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:11,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:13,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:15,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:17,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:29:19,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:29:19,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:21,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:23,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:25,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:27,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:29,298 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:29:29,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:31,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:33,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:29:34,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:29:35,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:37,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:39,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:41,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:43,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:45,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:47,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:49,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:29:49,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:29:51,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:53,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:55,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:57,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:29:59,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:29:59,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:01,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:03,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:04,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:30:04,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:30:05,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:07,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:09,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:11,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:13,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:16,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:18,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:20,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:30:20,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:30:20,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:22,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:24,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:26,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:28,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:29,438 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:30:30,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:32,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:34,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:35,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:30:35,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:30:36,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:38,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:40,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:42,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:44,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:46,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:30:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:30:50,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:30:59,507 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:31:06,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:31:06,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:31:21,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:31:21,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:31:29,577 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:31:36,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:31:36,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:31:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:31:47,840 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 06:31:47,892 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 06:31:47,969 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 06:31:48,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 06:31:48,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:31:50,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:31:52,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:31:52,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:31:54,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:31:59,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:32:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:32:07,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:32:10,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:32:22,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:32:22,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:32:29,731 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:32:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:32:37,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:32:52,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:32:52,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:32:59,806 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:33:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:33:08,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:33:23,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:33:23,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:33:29,881 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:33:38,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:33:38,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:33:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:33:53,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:33:59,958 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:34:09,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:34:09,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:34:24,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:34:24,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:34:30,035 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:34:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:34:39,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:34:54,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:34:54,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:35:00,114 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:35:10,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:35:10,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:35:25,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:35:25,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:35:30,192 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:35:40,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:35:40,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:35:55,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:35:55,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:36:00,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:36:04,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:04,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:04,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:08,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:10,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:10,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:11,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:36:11,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:36:11,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:11,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:12,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:14,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:16,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:17,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:17,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:19,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:21,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:21,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:22,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:23,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:25,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:26,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:36:26,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:36:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:26,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:27,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:29,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:30,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:36:31,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:33,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:33,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:35,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:35,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:36,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:37,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:39,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:41,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:36:41,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:36:41,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:42,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:43,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:44,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:45,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:47,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:49,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:49,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:49,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:51,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:53,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:36:53,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:55,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:36:56,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:36:56,913 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:37:00,394 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:37:03,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:09,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:12,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:37:12,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:37:13,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:17,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:21,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:21,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:21,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:25,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:37:27,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:37:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:28,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:29,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:30,460 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:37:31,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:31,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:35,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:36,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:36,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:37,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:39,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:41,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:37:42,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:37:43,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:45,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:47,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:49,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:50,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:51,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:55,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:57,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:37:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:37:58,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:37:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:37:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:00,532 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:38:00,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:02,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:03,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:03,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:04,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:06,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:08,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:08,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:10,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:11,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:12,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:13,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:38:13,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:38:14,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:15,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:16,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:18,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:18,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:19,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:19,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:20,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:21,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:22,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:24,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:26,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:28,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:38:28,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:38:30,606 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:38:30,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:32,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:32,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:34,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:36,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:36,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:37,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:38,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:38,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:39,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:40,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:42,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:43,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:43,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:38:43,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:38:44,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:46,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:47,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:47,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:48,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:49,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:50,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:52,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:38:58,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:38:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:38:59,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:39:00,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:00,681 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:39:06,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:10,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:14,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:39:14,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:39:14,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:18,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:22,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:26,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:28,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:29,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:39:29,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:39:30,755 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:39:30,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:30,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:32,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:35,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:36,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:37,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:38,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:38,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:41,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:42,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:44,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:45,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:39:45,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:39:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:48,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:50,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:52,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:56,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:56,877 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 06:39:57,088 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 06:39:57,164 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 06:39:57,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 06:39:57,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:57,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:58,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:39:59,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:39:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:00,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:40:00,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:40:00,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:01,175 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:40:03,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:05,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:07,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:07,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:09,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:11,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:12,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:13,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:15,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:15,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:40:15,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:40:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:16,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:17,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:19,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:19,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:19,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:20,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:21,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:22,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:22,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:23,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:25,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:27,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:29,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:30,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:40:30,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:40:31,250 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:40:31,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:33,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:33,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:34,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:35,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:37,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:39,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:39,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:41,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:42,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:43,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:45,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:45,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:40:46,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:40:47,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:40:47,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:49,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:40:59,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:01,323 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:41:01,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:41:01,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:41:03,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:04,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:04,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:08,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:09,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:13,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:41:16,687 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:41:17,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:19,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:23,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:27,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:29,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:31,397 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:41:31,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:41:31,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:41:33,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:33,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:35,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:37,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:39,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:42,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:42,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:46,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:41:47,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:41:48,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:49,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:52,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:53,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:53,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:56,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:41:57,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:41:58,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:00,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:01,469 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:42:01,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:01,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:02,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:42:02,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:42:02,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:04,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:06,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:08,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:09,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:09,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:10,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:11,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:12,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:14,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:14,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:16,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:42:17,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:42:18,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:18,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:20,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:20,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:22,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:24,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:25,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:25,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:26,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:28,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:30,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:31,541 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:42:32,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:32,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:33,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:42:33,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:42:33,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:34,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:36,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:36,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:37,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:37,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:38,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:40,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:40,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:42,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:44,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:46,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:42:48,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:42:48,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:42:50,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:42:56,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:00,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:01,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:01,616 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:43:03,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:43:03,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:43:04,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:06,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:08,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:12,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:16,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:18,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:43:18,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:43:18,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:22,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:26,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:28,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:31,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:43:32,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:32,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:33,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:43:33,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:43:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:34,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:35,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:35,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:36,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:40,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:42,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:42,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:44,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:46,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:47,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:48,389 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 06:43:48,599 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 06:43:48,721 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 06:43:48,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 06:43:48,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:49,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:43:49,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:43:50,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:51,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:52,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:54,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:54,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:43:56,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:43:58,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:00,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:00,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:00,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:01,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:44:02,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:04,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:44:04,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:44:04,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:04,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:05,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:06,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:08,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:08,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:09,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:10,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:12,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:13,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:14,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:14,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:16,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:18,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:19,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:44:19,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:44:20,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:20,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:20,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:21,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:22,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:23,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:23,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:24,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:26,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:29,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:30,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:31,886 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:44:32,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:33,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:33,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:34,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:34,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:44:34,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:44:35,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:35,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:36,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:36,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:38,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:44,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:44,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:47,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:48,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:49,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:50,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:44:50,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:44:51,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:55,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:44:56,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:56,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:44:59,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:01,956 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:45:03,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:05,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:05,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:45:05,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:45:06,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:07,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:11,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:13,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:17,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:21,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:45:21,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:45:21,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:23,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:27,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:29,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:31,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:32,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:45:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:32,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:34,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:35,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:36,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:45:36,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:45:37,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:38,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:38,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:39,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:39,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:40,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:41,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:42,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:42,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:43,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:46,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:46,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:47,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:49,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:51,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:45:51,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:45:51,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:51,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:52,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:52,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:53,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:55,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:57,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:45:57,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:45:59,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:01,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:01,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:02,100 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:46:03,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:03,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:04,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:04,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:05,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:06,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:46:06,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:46:07,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:08,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:09,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:11,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:12,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:12,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:13,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:14,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:15,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:17,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:17,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:18,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:19,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:20,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:20,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:21,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:22,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:46:22,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:46:23,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:24,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:25,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:26,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:27,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:29,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:29,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:30,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:30,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:31,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:32,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:46:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:46:37,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:46:38,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:41,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:45,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:46,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:48,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:49,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:52,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:52,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:46:52,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:46:53,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:57,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:46:59,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:46:59,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:01,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:02,243 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:47:03,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:07,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:07,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:47:07,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:47:07,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:12,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:14,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:18,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:20,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:47:23,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:47:24,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:26,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:28,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:31,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:32,306 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:47:32,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:34,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:34,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:35,070 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 06:47:35,277 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 06:47:35,341 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 06:47:35,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 06:47:35,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:36,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:38,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:47:38,421 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:47:38,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:39,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:39,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:40,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:42,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:44,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:45,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:45,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:46,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:47,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:48,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:50,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:52,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:54,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:47:54,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:47:54,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:55,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:47:56,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:47:58,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:00,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:00,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:02,404 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:48:02,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:04,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:04,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:05,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:06,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:07,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:07,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:08,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:48:09,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:48:10,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:10,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:11,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:12,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:14,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:16,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:18,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:20,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:22,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:22,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:23,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:23,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:24,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:25,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:48:25,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:48:25,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:25,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:26,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:28,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:30,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:32,476 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:48:38,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:39,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:40,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:48:40,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:48:44,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:44,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:46,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:48,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:54,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:55,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:48:55,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:48:56,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:48:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:48:57,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:00,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:01,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:02,550 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:49:02,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:06,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:49:10,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:49:10,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:12,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:12,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:16,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:18,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:20,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:24,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:49:26,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:49:26,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:28,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:29,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:49:32,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:32,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:33,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:34,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:36,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:37,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:37,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:38,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:40,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:49:41,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:49:41,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:41,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:44,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:46,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:46,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:48,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:48,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:48,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:50,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:52,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:52,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:54,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:55,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:55,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:49:56,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:49:56,757 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:49:56,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:49:58,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:00,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:02,695 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:50:02,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:04,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:05,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:05,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:06,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:08,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:08,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:10,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:12,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:50:12,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:50:12,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:12,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:13,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:14,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:15,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:15,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:16,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:18,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:18,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:20,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:21,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:22,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:23,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:23,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:24,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:27,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:27,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:50:27,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:50:32,769 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:50:32,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:32,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:37,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:37,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:39,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:39,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:41,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:42,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:50:42,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:50:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:43,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:44,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:44,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:47,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:49,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:53,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:56,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:56,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:57,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:50:57,945 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:50:57,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:50:58,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:58,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:50:59,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:00,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:02,843 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:51:03,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:04,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:04,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:05,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:09,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:09,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:11,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:13,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:51:13,183 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:51:15,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:17,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:19,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:23,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:25,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:25,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:26,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:26,291 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 06:51:26,515 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 06:51:26,593 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 06:51:26,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 06:51:27,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:28,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:51:28,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:51:28,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:29,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:29,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:30,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:31,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:32,935 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:51:33,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:35,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:37,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:39,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:40,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:40,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:41,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:43,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:51:43,742 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:51:43,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:45,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:45,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:46,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:46,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:47,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:49,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:49,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:51,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:52,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:52,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:53,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:55,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:51:57,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:51:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:51:59,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:51:59,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:00,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:00,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:01,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:02,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,012 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:52:03,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:03,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:04,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:04,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:05,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:07,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:08,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:08,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:09,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:10,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:10,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:11,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:14,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:14,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:52:14,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:52:16,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:16,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:19,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:29,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:52:29,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:52:29,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:29,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:33,086 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:52:35,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:36,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:36,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:39,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:40,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:43,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:43,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:44,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:52:44,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:52:45,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:46,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:47,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:51,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:51,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:55,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:57,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:52:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:58,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:52:59,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:53:00,002 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:53:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:00,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:01,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:03,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:53:05,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:07,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:09,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:11,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:11,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:13,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:14,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:14,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:15,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:15,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:53:15,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:53:19,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:21,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:23,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:23,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:25,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:29,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:30,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:53:30,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:53:31,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:32,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:32,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:33,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:33,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:53:34,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:34,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:35,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:37,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:37,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:39,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:41,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:43,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:45,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:53:45,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:53:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:47,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:47,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:49,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:50,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:51,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:53,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:55,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:55,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:57,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:58,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:53:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:53:59,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:54:00,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:54:01,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:03,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:03,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:54:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:04,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:05,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:05,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:06,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:07,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:07,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:09,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:09,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:11,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:12,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:13,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:13,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:15,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:16,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:54:16,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:54:17,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:24,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:24,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:25,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:29,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:31,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:31,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:54:31,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:54:33,383 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:54:35,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:40,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:43,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:43,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:44,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:45,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:45,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:54:46,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:54:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:46,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:48,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:48,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:50,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:52,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:52,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:55,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:56,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:58,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:54:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:54:59,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:02,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:02,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:55:02,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:55:03,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:03,456 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:55:04,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:06,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:08,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:10,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:12,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:14,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:16,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:17,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:55:17,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:55:18,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:20,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:20,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:20,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 06:55:21,536 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 06:55:21,746 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 06:55:21,839 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 06:55:22,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 06:55:23,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:24,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:26,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:28,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:30,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:32,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:32,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:55:32,913 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:55:33,792 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:55:34,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:36,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:38,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:40,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:42,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:44,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:46,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:48,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:48,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:55:48,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:55:50,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:52,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:54,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:56,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:55:58,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:00,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:02,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:03,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:56:03,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:56:03,869 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:56:04,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:06,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:08,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:10,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:12,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:14,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:17,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:18,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:56:18,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:56:19,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:21,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:23,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:25,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:27,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:29,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:31,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:33,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:33,945 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:56:34,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:56:34,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:56:35,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:37,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:39,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:41,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:43,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:45,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:47,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:49,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:49,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:56:49,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:56:51,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:53,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:55,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:57,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:56:59,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:01,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:03,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:04,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:57:04,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:57:04,626 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:57:05,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:07,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:09,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:11,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:13,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:15,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:17,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:19,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:19,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:57:19,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:57:21,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:23,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:25,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:27,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:29,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:31,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:33,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:34,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:57:35,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:35,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:57:35,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:57:37,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:39,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:42,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:44,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:46,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:48,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:50,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:57:50,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:57:52,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:54,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:56,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:57:58,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:00,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:02,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:04,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:04,170 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:58:06,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:58:06,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:58:06,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:08,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:10,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:12,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:14,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:16,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:18,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:20,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:58:21,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:58:22,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:24,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:26,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:28,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:30,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:32,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:34,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:34,246 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:58:36,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:58:36,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:58:38,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:40,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:42,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:44,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:46,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:48,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:51,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:58:51,896 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:58:53,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:55,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:57,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:58:59,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:01,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:03,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:04,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:59:05,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:07,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:07,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:59:07,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:59:09,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:11,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:13,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:15,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:17,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:19,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:21,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:59:22,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:59:23,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:25,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:27,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:29,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:31,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:33,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:34,395 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 06:59:35,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:37,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:37,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:59:37,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:59:39,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:41,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:43,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:45,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:47,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:49,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:51,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 06:59:52,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 06:59:53,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:55,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:57,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 06:59:59,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:01,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:03,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:04,470 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:00:05,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:07,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:08,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:00:08,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:00:09,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:12,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:14,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:16,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:18,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:20,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:22,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:23,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:00:23,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:00:24,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:26,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:28,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:30,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:32,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:34,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:34,542 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:00:36,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:38,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:38,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:00:38,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:00:40,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:42,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:44,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:46,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:48,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:50,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:52,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:54,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:00:54,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:00:54,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:56,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:00:58,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:00,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:02,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:04,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:04,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:01:06,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:08,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:09,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:01:09,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:01:10,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:12,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:14,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:16,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:18,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:20,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:22,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:24,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:26,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:27,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:01:27,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:01:28,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:30,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:32,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:34,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:34,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:01:36,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:39,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:41,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:42,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:01:42,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:01:43,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:45,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:47,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:49,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:51,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:53,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:55,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:57,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:01:58,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:01:58,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:01:59,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:01,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:03,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:04,766 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:02:05,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:07,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:09,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:11,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:13,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:13,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:02:13,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:02:15,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:17,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:19,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:21,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:23,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:25,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:27,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:28,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:02:28,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:02:29,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:31,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:33,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:34,841 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:02:35,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:37,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:39,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:41,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:43,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:44,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:02:44,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:02:45,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:47,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:49,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:51,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:53,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:55,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:57,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:59,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:02:59,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:02:59,375 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:03:01,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:03,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:04,913 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:03:05,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:07,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:09,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:11,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:13,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:14,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:03:14,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:03:15,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:17,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:19,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:21,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:23,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:25,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:27,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:29,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:03:29,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:03:31,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:33,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:34,990 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:03:35,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:38,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:40,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:42,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:44,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:45,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:03:45,109 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:03:46,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:48,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:50,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:52,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:54,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:56,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:03:58,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:00,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:04:00,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:04:02,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:04,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:05,067 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:04:06,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:08,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:10,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:12,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:14,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:04:16,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:04:16,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:18,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:20,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:22,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:24,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:26,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:28,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:30,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:31,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:04:31,428 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:04:32,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:34,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:35,145 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:04:36,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:38,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:40,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:42,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:44,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:46,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:46,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:04:46,740 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:04:48,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:50,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:52,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:54,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:56,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:04:58,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:00,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:05:01,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:05:02,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:04,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:05,227 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:05:06,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:08,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:10,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:12,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:14,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:16,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:17,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:05:17,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:05:18,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:20,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:22,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:25,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:27,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:29,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:31,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:32,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:05:32,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:05:33,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:35,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:05:35,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:37,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:39,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:41,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:43,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:45,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:47,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:05:47,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:05:49,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:51,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:53,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:55,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:57,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:05:59,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:01,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:03,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:06:03,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:06:03,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:05,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:05,376 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:06:07,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:09,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:11,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:13,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:15,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:17,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:18,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:06:18,363 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:06:19,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:21,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:23,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:25,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:27,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:29,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:31,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:33,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:33,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:06:33,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:06:35,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:35,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:06:37,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:39,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:41,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:43,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:45,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:47,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:48,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:06:48,867 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:06:49,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:51,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:53,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:55,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:57,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:06:59,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:01,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:03,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:04,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:07:04,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:07:05,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:05,527 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:07:07,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:09,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:11,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:13,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:16,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:18,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:07:19,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:07:20,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:22,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:24,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:26,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:28,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:30,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:32,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:34,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:34,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:07:34,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:07:35,603 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:07:36,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:38,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:40,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:42,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:44,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:46,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:48,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:07:50,037 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:07:50,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:52,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:54,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:56,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:07:58,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:00,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:02,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:04,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:05,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:08:05,375 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:08:05,677 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:08:06,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:08,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:10,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:12,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:14,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:17,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:19,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:20,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:08:20,653 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:08:21,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:23,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:25,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:27,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:29,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:31,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:33,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:35,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:35,755 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:08:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:08:35,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:08:37,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:39,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:41,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:43,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:45,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:47,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:49,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:08:51,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:08:51,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:53,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:55,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:57,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:08:59,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:01,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:03,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:05,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:05,830 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:09:06,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:09:06,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:09:07,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:09,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:11,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:13,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:15,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:17,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:20,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:09:21,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:09:22,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:24,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:26,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:28,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:30,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:32,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:34,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:35,907 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:09:36,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:36,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:09:36,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:09:38,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:40,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:42,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:44,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:46,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:48,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:50,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:09:52,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:09:52,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:54,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:56,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:09:58,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:00,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:02,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:04,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:05,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:10:06,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:07,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:10:07,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:10:08,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:10,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:12,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:14,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:16,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:18,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:20,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:22,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:22,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:10:22,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:10:24,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:26,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:28,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:30,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:32,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:34,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:36,056 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:10:36,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:38,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:10:38,006 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:10:38,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:40,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:42,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:44,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:47,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:49,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:51,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:53,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:10:53,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:10:53,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:55,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:57,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:10:59,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:01,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:03,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:05,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:06,132 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:11:07,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:08,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:11:08,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:11:09,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:11,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:13,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:15,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:17,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:19,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:21,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:23,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:11:23,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:11:25,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:27,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:29,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:31,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:33,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:35,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:36,207 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:11:37,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:11:39,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:11:39,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:41,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:43,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:45,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:47,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:49,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:51,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:53,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:54,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:11:54,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:11:55,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:57,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:11:59,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:01,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:03,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:05,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:06,279 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:12:07,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:09,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:12:09,553 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:12:11,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:13,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:15,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:17,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:20,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:22,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:24,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:12:24,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:12:26,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:28,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:30,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:32,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:34,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:36,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:12:36,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:38,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:40,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:12:40,100 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:12:40,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:42,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:44,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:46,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:48,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:50,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:52,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:54,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:55,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:12:55,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:12:56,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:12:58,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:00,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:02,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:04,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:06,426 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:13:06,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:08,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:10,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:10,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:13:10,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:13:12,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:14,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:16,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:18,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:20,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:23,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:25,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:26,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:13:26,029 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:13:27,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:29,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:31,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:33,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:35,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:36,490 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:13:37,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:39,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:41,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:13:41,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:13:41,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:43,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:45,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:47,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:49,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:51,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:53,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:55,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:56,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:13:56,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:13:57,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:13:59,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:01,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:03,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:05,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:06,563 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:14:07,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:09,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:11,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:11,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:14:11,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:14:13,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:15,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:17,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:19,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:21,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:23,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:25,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:27,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:14:27,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:14:27,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:29,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:31,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:33,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:36,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:36,638 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:14:38,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:40,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:14:42,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:14:42,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:44,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:46,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:48,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:50,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:52,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:54,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:56,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:14:57,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:14:57,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:14:58,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:00,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:02,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:04,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:06,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:06,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:15:08,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:10,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:12,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:15:13,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:15:14,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:16,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:18,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:20,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:22,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:24,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:26,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:28,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:15:28,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:15:28,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:30,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:32,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:34,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:36,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:36,788 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:15:38,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:40,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:42,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:15:43,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:15:44,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:46,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:49,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:51,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:53,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:55,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:57,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:15:58,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:15:58,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:15:59,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:01,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:03,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:05,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:06,864 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:16:07,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:09,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:11,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:13,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:16:14,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:16:15,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:17,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:19,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:21,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:23,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:25,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:27,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:29,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:16:29,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:16:29,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:31,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:33,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:35,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:36,940 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:16:37,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:39,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:41,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:43,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:16:44,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:16:45,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:47,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:49,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:51,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:53,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:55,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:57,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:59,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:16:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:16:59,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:17:01,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:04,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:06,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:07,014 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:17:08,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:10,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:12,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:14,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:15,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:17:15,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:17:16,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:18,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:20,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:22,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:24,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:26,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:28,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:30,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:17:30,597 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:17:30,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:32,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:34,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:36,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:37,088 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:17:38,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:40,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:42,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:44,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:46,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:17:46,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:17:46,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:48,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:50,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:52,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:54,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:56,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:17:58,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:00,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:01,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:18:01,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:18:02,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:04,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:06,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:07,168 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:18:08,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:10,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:12,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:14,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:16,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:16,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:18:16,894 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:18:18,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:20,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:22,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:24,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:26,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:28,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:30,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:18:32,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:18:32,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:34,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:36,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:37,242 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:18:39,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:41,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:43,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:45,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:18:47,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:18:47,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:49,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:51,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:53,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:55,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:57,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:18:59,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:01,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:02,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:19:02,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:19:03,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:05,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:07,317 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:19:07,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:09,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:11,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:13,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:15,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:17,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:17,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:19:17,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:19:19,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:21,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:23,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:25,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:27,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:29,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:31,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:33,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:19:33,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:19:34,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:36,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:37,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:19:38,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:40,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:42,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:44,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:46,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:19:48,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:19:48,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:50,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:52,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:54,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:56,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:19:58,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:00,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:02,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:03,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:20:03,866 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:20:04,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:06,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:07,462 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:20:08,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:10,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:12,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:14,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:16,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:18,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:19,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:20:19,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:20:20,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:22,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:24,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:26,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:28,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:30,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:32,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:34,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:20:34,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:20:36,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:37,531 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:20:38,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:40,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:42,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:44,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:46,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:48,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:50,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:20:50,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:20:50,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:52,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:54,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:56,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:20:58,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:01,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:03,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:05,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:21:05,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:21:05,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:07,597 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:21:07,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:09,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:11,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:13,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:15,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:17,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:19,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:20,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:21:20,889 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:21:21,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:23,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:25,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:27,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:29,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:31,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:33,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:35,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:36,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:21:36,113 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:21:37,671 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:21:37,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:39,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:41,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:43,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:45,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:47,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:49,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:51,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:21:51,373 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:21:51,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:53,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:55,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:57,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:21:59,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:01,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:03,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:05,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:22:06,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:22:07,741 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:22:07,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:09,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:11,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:13,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:15,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:17,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:19,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:21,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:22,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:22:22,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:22:23,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:25,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:27,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:29,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:31,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:33,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:35,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:37,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:22:37,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:22:37,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:37,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:22:39,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:41,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:44,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:46,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:48,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:50,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:52,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:22:52,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:22:52,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:54,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:56,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:22:58,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:00,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:02,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:04,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:06,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:07,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:23:07,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:23:08,068 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:23:08,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:10,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:12,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:14,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:16,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:18,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:20,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:22,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:23:23,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:23:24,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:26,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:28,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:30,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:32,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:34,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:36,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:37,962 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:23:38,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:23:38,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:23:38,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:40,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:42,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:44,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:46,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:48,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:50,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:52,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:53,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:23:53,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:23:54,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:56,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:23:58,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:00,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:02,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:04,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:06,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:08,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:24:09,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:24:09,113 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:24:09,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:11,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:13,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:15,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:17,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:19,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:21,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:23,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:24,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:24:24,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:24:25,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:27,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:29,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:31,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:33,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:35,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:37,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:38,102 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:24:39,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:24:39,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:24:39,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:41,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:43,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:45,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:47,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:49,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:51,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:53,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:54,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:24:54,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:24:55,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:57,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:24:59,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:01,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:03,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:05,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:07,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:08,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:25:09,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:10,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:25:10,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:25:11,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:13,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:15,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:17,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:19,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:21,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:23,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:25,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:25:25,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:25:25,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:27,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:29,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:31,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:33,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:35,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:37,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:38,239 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:25:39,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:40,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:25:40,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:25:42,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:44,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:46,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:48,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:50,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:52,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:54,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:55,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:25:55,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:25:56,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:25:58,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:00,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:02,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:04,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:06,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:08,309 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:26:08,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:10,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:11,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:26:11,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:26:12,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:14,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:16,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:18,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:20,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:22,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:24,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:26,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:26:26,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:26:26,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:28,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:30,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:32,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:34,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:36,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:38,383 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:26:38,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:40,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:41,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:26:41,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:26:42,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:44,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:46,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:48,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:50,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:52,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:54,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:56,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:26:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:26:57,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:26:58,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:00,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:02,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:04,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:06,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:08,447 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:27:08,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:10,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:27:12,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:27:13,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:15,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:17,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:19,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:21,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:23,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:25,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:27,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:27:27,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:27:27,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:29,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:31,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:33,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:35,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:37,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:38,503 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:27:39,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:41,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:27:42,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:27:43,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:45,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:47,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:49,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:51,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:53,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:55,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:57,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:27:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:27:58,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:27:59,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:01,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:03,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:05,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:07,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:08,579 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:28:09,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:11,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:28:13,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:28:13,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:15,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:17,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:19,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:21,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:23,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:25,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:27,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:28:28,647 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:28:29,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:31,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:33,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:35,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:37,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:38,654 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:28:39,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:41,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:28:43,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:28:43,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:45,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:47,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:49,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:51,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:53,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:55,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:57,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:28:59,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:28:59,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:28:59,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:01,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:03,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:05,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:07,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:08,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:29:09,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:11,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:14,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:29:14,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:29:14,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:16,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:18,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:20,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:22,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:24,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:26,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:28,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:29,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:29:29,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:29:30,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:32,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:34,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:36,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:38,803 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:29:38,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:40,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:42,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:44,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:44,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:29:44,969 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:29:46,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:48,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:50,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:52,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:54,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:56,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:29:58,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:00,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:30:00,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:30:00,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:02,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:04,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:06,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:08,877 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:30:08,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:10,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:12,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:14,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:15,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:30:15,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:30:16,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:18,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:20,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:22,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:24,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:26,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:28,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:30,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:30:30,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:30:30,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:32,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:34,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:36,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:38,956 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:30:38,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:40,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:42,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:44,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:46,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:30:46,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:30:46,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:48,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:51,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:53,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:55,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:57,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:30:59,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:01,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:31:01,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:31:01,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:03,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:05,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:07,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:09,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:31:09,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:12,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:14,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:16,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:16,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:31:16,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:31:18,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:20,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:22,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:24,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:26,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:28,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:30,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:31,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:31:31,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:31:32,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:34,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:36,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:38,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:39,108 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:31:40,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:42,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:44,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:46,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:47,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:31:47,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:31:48,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:50,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:52,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:54,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:56,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:31:58,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:00,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:02,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:02,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:32:02,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:32:04,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:06,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:08,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:09,186 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:32:10,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:12,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:14,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:16,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:17,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:32:17,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:32:18,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:20,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:22,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:24,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:26,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:28,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:30,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:32,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:32:33,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:32:34,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:36,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:38,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:39,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:32:40,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:43,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:45,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:47,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:48,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:32:48,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:32:49,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:51,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:53,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:55,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:57,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:32:59,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:01,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:03,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:03,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:33:03,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:33:05,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:07,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:09,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:09,338 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:33:11,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:13,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:15,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:17,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:18,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:33:18,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:33:19,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:21,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:23,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:25,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:27,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:29,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:31,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:33,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:34,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:33:34,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:33:35,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:37,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:39,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:39,414 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:33:41,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:43,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:45,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:47,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:49,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:49,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:33:49,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:33:51,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:53,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:55,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:57,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:33:59,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:01,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:03,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:04,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:34:04,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:34:05,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:07,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:09,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:09,491 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:34:11,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:14,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:16,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:18,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:20,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:20,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:34:20,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:34:22,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:24,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:26,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:28,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:30,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:32,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:34,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:35,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:34:35,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:34:36,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:38,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:39,565 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:34:40,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:42,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:44,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:46,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:48,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:50,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:34:50,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:34:52,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:54,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:56,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:34:58,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:00,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:02,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:04,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:06,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:35:06,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:35:08,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:09,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:35:10,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:12,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:14,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:16,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:18,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:20,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:35:21,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:35:22,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:24,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:26,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:28,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:30,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:32,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:34,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:36,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:36,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:35:36,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:35:39,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:39,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:35:41,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:43,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:45,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:47,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:49,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:51,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:35:51,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:35:53,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:55,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:57,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:35:59,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:01,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:03,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:05,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:07,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:07,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:36:07,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:36:09,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:09,787 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:36:11,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:13,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:15,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:17,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:19,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:21,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:22,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:36:22,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:36:23,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:25,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:27,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:29,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:31,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:33,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:35,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:37,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:37,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:36:37,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:36:39,859 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:36:40,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:42,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:44,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:46,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:48,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:50,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:52,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:53,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:36:53,248 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:36:54,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:56,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:36:58,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:00,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:02,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:04,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:06,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:08,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:37:08,566 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:37:09,935 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:37:10,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:12,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:14,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:16,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:37:23,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:37:23,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:37:39,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:37:39,083 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:37:40,005 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:37:54,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:37:54,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:38:09,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:38:09,627 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:38:10,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:38:15,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:38:15,354 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 07:38:15,406 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 07:38:15,485 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 07:38:16,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 07:38:16,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:38:17,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:38:22,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:38:24,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:38:24,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:38:40,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:38:40,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:38:40,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:38:40,350 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:38:55,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:38:55,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:39:10,243 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:39:10,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:39:10,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:39:25,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:39:25,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:39:40,318 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:39:41,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:39:41,029 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:39:56,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:39:56,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:40:10,397 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:40:11,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:40:11,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:40:26,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:40:26,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:40:40,476 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:40:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:40:41,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:40:57,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:40:57,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:41:10,554 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:41:12,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:41:12,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:41:27,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:41:27,878 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:41:40,632 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:41:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:41:43,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:41:58,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:41:58,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:42:10,710 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:42:13,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:42:13,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:42:28,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:42:28,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:42:34,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:37,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:40,790 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:42:41,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:43,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:44,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:42:44,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:42:45,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:47,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:47,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:47,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:48,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:49,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:49,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:50,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:51,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:51,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:52,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:53,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:54,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:54,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:55,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:57,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:58,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:58,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:42:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:42:59,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:42:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:42:59,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:01,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:02,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:03,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:05,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:07,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:08,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:09,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:09,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:10,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:43:11,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:13,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:14,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:43:14,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:43:15,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:16,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:17,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:17,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:19,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:21,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:23,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:23,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:24,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:25,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:27,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:43:30,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:43:31,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:35,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:35,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:40,933 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:43:41,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:44,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:45,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:45,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:43:45,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:43:46,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:46,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:49,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:53,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:55,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:43:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:56,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:43:59,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:44:00,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:44:01,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:01,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:03,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:03,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:05,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:09,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:09,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:44:11,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:11,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:15,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:16,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:44:16,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:44:16,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:17,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:19,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:20,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:23,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:25,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:27,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:27,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:27,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:27,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:27,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:30,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:31,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:31,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:44:31,456 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:44:35,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:37,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:39,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:41,074 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:44:41,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:43,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:45,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:46,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:44:46,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:44:47,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:47,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:48,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:49,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:51,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:53,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:53,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:53,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:54,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:55,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:44:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:57,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:44:58,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:00,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:01,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:45:01,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:45:02,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:04,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:06,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:07,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:07,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:08,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:10,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:11,148 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:45:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:12,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:14,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:15,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:16,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:16,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:45:17,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:45:17,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:17,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:18,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:20,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:22,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:24,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:24,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:28,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:32,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:32,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:45:32,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:45:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:33,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:36,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:40,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:40,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:40,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:41,220 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:45:44,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:47,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:45:47,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:45:48,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:50,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:50,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:52,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:53,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:53,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:56,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:45:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:45:58,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:00,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:02,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:02,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:46:02,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:46:03,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:03,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:06,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:08,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:08,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:11,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:46:12,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:13,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:14,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:15,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:15,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:16,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:18,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:46:18,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:46:20,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:22,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:23,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:24,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:25,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:25,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,251 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 07:46:26,466 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 07:46:26,536 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 07:46:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:26,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:27,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 07:46:28,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:30,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:32,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:32,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:46:33,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:46:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:34,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:38,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:40,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:41,391 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:46:42,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:44,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:46,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:48,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:48,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:46:48,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:46:50,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:52,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:52,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:53,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:53,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:54,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:55,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:55,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:56,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:46:57,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:57,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:46:58,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:00,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:01,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:02,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:04,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:47:04,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:47:04,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:05,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:06,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:06,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:08,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:08,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:10,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:11,462 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:47:11,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:11,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:12,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:13,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:15,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:15,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:17,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:19,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:47:19,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:47:19,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:20,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:20,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:29,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:30,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:30,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:33,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:47:34,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:47:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:36,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:37,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:38,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:38,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:41,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:41,538 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:47:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:44,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:45,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:49,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:47:49,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:47:50,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:51,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:55,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:47:56,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:47:59,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:01,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:03,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:48:05,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:48:05,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:07,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:10,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:10,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:11,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:11,614 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:48:13,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:15,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:17,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:48:20,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:48:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:21,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:21,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:22,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:23,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:25,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:25,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:27,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:28,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:29,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:31,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:33,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:34,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:35,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:35,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:48:35,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:48:37,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:39,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:41,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:41,691 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:48:43,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:45,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:47,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:48,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:48,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:49,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:50,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:48:50,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:48:51,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:53,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:53,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:55,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:57,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:59,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:48:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:48:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:00,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:01,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:03,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:05,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:06,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:49:06,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:49:07,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:09,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:09,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:10,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:11,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:11,762 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:49:13,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:21,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:49:21,696 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:49:23,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:27,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:29,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:31,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:32,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:35,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:35,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:36,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:49:36,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:49:37,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:37,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:39,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:41,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:49:43,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:46,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:48,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:48,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:50,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:52,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:49:52,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:49:54,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:49:56,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:49:56,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:00,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:02,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:04,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:06,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:50:07,498 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:50:08,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:10,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:11,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:50:13,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:13,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:14,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:16,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:16,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:16,933 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 07:50:17,155 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 07:50:17,222 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 07:50:17,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 07:50:17,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:17,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:18,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:19,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:20,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:20,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:22,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:50:22,835 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:50:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:23,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:24,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:24,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:26,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:28,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:30,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:32,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:34,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:36,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:50:38,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:50:38,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:40,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:42,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:50:42,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:44,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:46,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:46,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:46,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:48,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:48,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:49,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:50,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:51,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:51,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:52,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:53,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:50:53,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:50:54,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:54,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:56,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:57,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:57,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:58,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:50:59,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:50:59,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:00,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:02,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:02,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:03,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:04,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:06,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:07,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:07,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:08,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:51:08,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:51:08,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:12,090 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:51:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:16,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:18,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:21,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:22,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:51:23,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:51:26,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:26,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:29,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:30,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:31,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:33,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:34,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:38,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:38,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:39,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:51:39,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:51:41,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,167 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:51:42,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:42,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:43,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:44,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:46,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:48,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:51,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:51:54,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:51:55,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:57,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:51:57,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:51:57,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:01,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:02,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:02,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:03,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:07,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:09,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:09,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:52:09,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:52:11,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:11,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:11,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:52:12,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:13,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:14,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:15,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:17,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:18,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:18,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:19,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:21,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:21,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:23,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:25,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:52:25,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:52:25,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:27,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:28,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:29,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:31,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:31,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:33,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:34,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:34,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:35,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:37,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:39,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:40,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:52:40,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:52:41,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:42,305 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:52:43,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:45,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:45,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:45,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:47,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:49,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:50,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:51,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:53,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:53,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:53,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:55,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:52:55,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:52:55,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:55,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:56,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:52:57,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:52:59,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:00,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:01,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:01,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:03,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:05,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:11,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:53:11,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:53:11,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:11,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:12,380 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:53:13,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:14,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:14,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:19,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:21,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:23,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:25,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:53:26,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:53:27,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:28,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:30,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:31,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:34,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:36,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:37,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:38,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:53:41,720 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:53:42,448 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:53:42,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:43,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:43,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:44,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:47,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:47,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:48,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:49,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:49,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:53:50,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:54,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:56,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:53:56,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:53:56,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:53:58,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:00,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:00,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:01,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:02,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:03,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:04,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:06,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:06,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:06,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:07,487 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 07:54:07,681 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 07:54:07,749 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 07:54:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:08,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 07:54:08,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:09,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:10,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:11,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:54:12,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:54:12,560 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:54:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:12,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:13,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:14,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:15,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:16,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:18,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:19,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:20,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:22,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:22,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:22,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:24,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:26,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:27,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:54:27,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:54:28,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:29,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:29,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:30,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:32,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:34,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:36,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:38,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:40,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:42,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:54:42,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:54:42,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:54:44,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:44,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:44,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:46,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:48,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:48,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:49,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:50,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:51,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:51,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:52,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:53,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:53,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:54,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:54,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:55,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:55,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:55,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:55,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:55,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:55,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:56,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:54:58,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:54:58,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:54:58,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:54:58,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:00,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:08,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:12,699 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:55:13,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:55:13,387 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:55:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:14,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:18,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:20,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:20,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:22,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:24,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:26,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:27,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:27,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:28,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:55:28,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:55:30,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:32,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:33,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:34,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:36,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:40,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:42,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:42,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:55:43,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:43,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:55:44,035 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:55:44,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:48,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:50,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:51,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:53,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:55:54,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:56,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:58,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:55:59,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:55:59,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:56:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:00,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:02,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:02,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:04,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:06,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:07,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:07,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:08,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:09,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:10,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:12,841 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:56:13,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:13,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:14,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:56:14,632 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:56:17,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:19,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:21,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:22,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:22,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:23,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:25,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:25,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:27,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:29,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:29,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:56:29,932 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:56:31,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:33,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:35,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:37,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:39,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:41,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:42,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:42,913 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:56:43,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:44,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:44,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:45,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:56:45,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:56:45,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:46,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:46,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:47,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:49,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:50,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:51,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:53,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:55,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:56:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:56:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:00,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:57:00,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:57:03,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:05,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:07,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:11,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:12,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:57:13,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:13,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:15,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:57:15,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:57:15,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:19,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:21,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:22,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:23,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:26,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:26,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:27,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:29,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:30,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:57:30,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:57:32,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:33,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:35,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:39,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:39,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:39,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:40,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:40,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:40,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:40,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:41,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:42,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:42,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:43,056 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:57:45,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:46,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:57:46,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:57:47,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:47,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:49,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:50,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:50,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:51,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:54,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:55,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:55,948 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 07:57:56,164 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 07:57:56,237 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 07:57:56,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 07:57:56,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:57,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:57,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:57:58,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:58,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:57:59,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:01,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:01,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:58:01,423 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:58:01,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:03,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:05,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:06,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:07,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:08,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:09,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:10,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:10,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:11,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:13,180 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:58:13,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:15,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:16,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:58:16,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:58:17,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:18,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:18,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:19,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:21,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:22,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:22,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:23,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:25,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:27,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:28,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:29,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:31,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:32,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:58:32,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:58:33,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:35,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:36,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:37,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:38,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:39,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:39,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:40,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:41,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,258 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:58:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:43,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:46,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:47,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:58:47,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:58:54,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:54,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:56,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:58:58,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:58:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:00,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:01,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:01,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:02,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:59:02,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:59:06,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:08,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:12,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:13,333 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:59:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:16,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:17,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:59:17,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:59:18,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:20,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:22,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:22,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:23,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:24,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:26,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:27,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:28,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:30,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:32,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:33,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:59:33,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:59:34,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:36,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:40,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:42,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:43,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 07:59:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:44,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:46,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 07:59:48,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 07:59:48,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:48,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:49,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:50,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:51,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:51,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:52,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:54,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:55,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 07:59:56,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 07:59:58,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:00,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:02,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:02,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:03,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:00:03,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:00:04,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:04,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:05,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:06,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:07,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:08,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:09,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:11,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:13,485 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:00:13,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:14,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:14,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:15,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:17,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:17,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:18,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:00:18,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:00:19,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:21,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:23,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:25,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:25,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:27,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:29,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:31,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:31,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:33,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:34,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:00:34,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:00:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:35,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:36,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:36,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:37,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:39,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:43,561 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:00:44,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:44,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:47,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:49,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:00:49,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:00:49,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:49,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:53,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:54,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:57,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:00:59,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:00:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:01,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:04,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:01:04,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:01:05,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:09,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:11,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:13,643 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:01:15,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:16,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:17,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:17,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:18,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:19,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:01:19,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:01:20,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:20,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:21,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:23,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:23,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:27,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:29,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:29,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:30,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:30,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:31,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:32,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:32,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:34,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:01:35,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:01:35,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:35,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:37,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:37,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:38,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:38,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:39,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:40,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:01:41,086 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 08:01:41,278 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 08:01:41,335 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 08:01:41,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 08:01:41,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:43,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:01:43,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:45,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:47,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:49,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:50,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:01:50,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:01:51,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:53,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:55,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:57,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:01:59,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:01,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:03,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:05,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:06,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:02:06,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:02:07,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:09,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:11,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:13,833 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:02:13,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:16,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:18,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:20,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:21,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:02:21,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:02:22,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:24,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:26,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:28,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:30,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:32,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:34,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:02:36,573 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:02:36,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:38,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:40,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:42,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:43,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:02:44,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:46,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:48,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:50,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:51,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:02:51,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:02:52,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:54,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:56,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:02:58,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:00,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:02,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:04,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:06,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:07,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:03:07,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:03:08,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:10,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:12,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:13,983 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:03:14,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:16,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:18,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:20,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:22,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:03:22,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:03:22,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:24,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:26,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:28,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:30,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:32,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:34,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:36,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:37,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:03:37,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:03:38,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:40,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:42,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:44,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:03:44,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:46,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:48,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:50,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:52,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:03:52,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:03:52,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:54,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:56,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:03:58,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:00,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:02,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:04,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:06,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:04:08,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:04:09,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:11,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:13,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:14,136 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:04:15,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:17,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:19,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:21,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:23,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:04:23,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:04:23,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:25,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:27,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:29,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:31,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:33,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:35,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:37,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:38,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:04:38,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:04:39,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:41,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:43,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:44,213 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:04:45,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:47,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:49,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:51,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:04:53,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:04:53,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:55,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:57,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:04:59,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:01,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:03,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:05,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:07,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:09,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:05:09,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:05:10,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:12,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:14,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:14,292 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:05:16,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:18,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:20,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:23,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:24,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:05:24,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:05:25,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:27,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:29,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:31,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:33,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:35,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:37,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:39,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:39,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:05:39,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:05:41,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:43,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:44,367 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:05:45,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:47,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:49,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:51,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:53,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:54,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:05:54,978 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:05:55,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:57,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:05:59,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:01,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:03,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:05,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:07,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:09,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:06:10,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:06:11,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:13,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:14,442 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:06:15,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:17,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:19,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:21,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:23,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:25,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:25,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:06:25,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:06:27,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:29,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:31,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:33,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:35,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:38,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:40,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:06:40,864 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:06:42,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:44,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:44,517 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:06:46,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:48,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:50,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:52,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:54,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:56,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:06:56,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:06:56,100 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:06:58,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:00,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:02,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:04,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:06,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:08,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:10,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:11,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:07:11,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:07:12,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:14,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:14,592 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:07:16,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:18,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:20,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:22,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:24,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:26,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:26,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:07:26,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:07:28,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:30,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:32,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:34,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:36,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:39,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:41,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:41,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:07:41,896 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:07:43,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:44,665 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:07:45,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:47,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:49,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:51,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:53,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:55,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:57,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:07:57,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:07:57,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:07:59,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:01,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:03,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:05,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:07,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:09,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:11,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:12,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:08:12,411 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:08:13,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:14,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:08:15,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:17,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:19,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:21,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:23,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:25,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:27,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:08:27,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:08:29,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:31,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:33,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:35,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:37,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:39,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:41,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:42,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:08:42,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:08:43,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:44,813 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:08:45,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:47,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:49,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:51,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:53,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:55,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:57,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:08:58,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:08:58,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:08:59,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:02,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:04,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:06,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:08,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:10,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:12,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:13,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:09:13,388 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:09:14,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:14,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:09:16,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:18,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:20,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:22,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:24,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:26,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:28,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:28,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:09:28,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:09:30,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:32,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:34,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:36,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:38,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:40,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:42,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:09:43,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:09:44,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:44,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:09:46,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:48,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:50,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:52,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:54,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:56,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:58,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:09:59,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:09:59,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:10:00,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:02,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:04,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:06,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:08,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:10,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:12,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:14,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:14,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:10:14,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:10:15,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:10:16,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:18,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:21,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:23,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:25,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:27,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:29,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:29,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:10:29,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:10:31,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:33,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:35,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:37,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:39,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:41,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:43,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:10:45,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:10:45,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:45,302 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:10:47,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:49,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:51,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:53,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:55,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:57,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:10:59,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:11:00,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:11:01,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:03,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:05,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:07,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:09,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:11,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:13,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:15,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:15,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:11:15,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:11:15,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:11:17,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:19,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:21,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:23,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:25,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:27,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:29,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:30,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:11:30,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:11:31,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:33,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:35,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:37,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:39,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:42,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:44,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:45,271 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:11:46,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:46,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:11:46,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:11:48,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:50,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:52,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:54,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:56,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:11:58,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:00,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:01,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:12:01,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:12:02,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:04,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:06,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:08,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:10,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:12,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:14,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:15,345 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:12:16,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:12:16,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:12:18,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:20,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:22,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:24,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:26,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:28,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:30,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:12:31,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:12:32,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:34,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:36,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:38,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:40,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:42,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:44,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:45,427 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:12:46,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:12:47,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:12:48,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:50,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:52,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:54,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:56,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:12:58,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:00,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:02,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:13:02,518 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:13:05,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:07,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:09,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:11,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:13,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:15,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:15,504 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:13:17,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:13:17,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:13:19,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:21,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:23,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:25,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:27,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:29,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:31,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:33,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:13:33,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:13:33,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:35,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:37,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:39,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:41,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:43,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:45,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:45,577 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:13:47,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:48,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:13:48,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:13:49,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:51,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:53,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:55,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:57,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:13:59,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:01,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:03,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:03,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:14:03,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:14:05,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:07,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:09,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:11,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:13,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:15,654 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:14:16,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:18,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:18,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:14:18,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:14:20,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:22,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:24,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:26,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:28,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:30,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:32,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:34,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:14:34,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:14:34,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:36,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:38,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:40,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:42,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:44,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:45,722 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:14:46,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:48,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:49,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:14:49,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:14:50,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:52,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:54,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:56,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:14:58,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:00,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:02,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:04,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:04,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:15:04,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:15:06,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:08,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:10,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:12,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:14,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:15,786 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:15:16,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:18,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:20,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:15:20,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:15:20,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:22,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:24,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:26,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:28,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:30,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:33,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:35,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:35,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:15:35,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:15:37,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:39,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:41,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:43,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:45,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:45,854 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:15:47,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:49,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:50,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:15:50,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:15:51,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:53,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:55,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:57,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:15:59,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:01,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:03,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:05,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:05,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:16:05,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:16:07,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:09,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:11,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:13,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:15,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:15,916 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:16:17,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:19,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:21,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:16:21,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:16:21,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:23,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:25,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:27,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:29,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:31,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:33,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:35,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:36,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:16:36,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:16:37,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:39,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:41,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:43,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:45,974 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:16:46,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:48,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:50,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:16:51,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:16:52,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:54,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:56,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:16:58,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:00,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:02,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:04,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:06,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:07,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:17:07,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:17:08,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:10,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:12,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:14,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:16,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:17:16,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:18,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:20,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:22,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:22,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:17:22,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:17:24,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:26,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:28,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:30,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:32,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:34,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:36,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:37,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:17:37,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:17:38,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:40,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:42,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:44,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:46,117 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:17:46,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:48,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:50,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:52,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:17:53,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:17:54,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:56,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:17:58,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:00,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:02,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:04,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:06,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:18:08,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:18:08,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:10,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:12,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:14,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:16,183 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:18:16,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:18,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:20,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:22,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:23,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:18:23,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:18:24,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:26,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:28,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:31,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:33,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:35,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:37,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:38,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:18:38,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:18:39,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:41,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:43,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:45,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:46,247 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:18:47,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:49,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:51,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:53,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:54,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:18:54,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:18:55,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:57,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:18:59,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:01,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:03,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:05,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:07,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:09,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:09,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:19:09,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:19:11,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:13,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:15,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:16,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:19:17,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:19,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:21,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:23,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:24,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:19:24,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:19:25,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:27,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:29,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:31,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:33,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:35,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:37,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:39,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:39,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:19:39,953 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:19:41,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:43,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:45,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:46,398 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:19:47,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:49,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:51,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:53,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:55,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:19:55,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:19:55,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:57,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:19:59,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:01,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:04,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:06,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:08,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:10,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:10,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:20:10,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:20:12,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:14,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:16,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:16,474 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:20:18,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:20,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:22,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:24,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:25,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:20:25,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:20:26,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:28,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:30,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:32,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:34,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:36,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:38,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:40,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:41,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:20:41,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:20:42,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:44,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:46,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:46,549 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:20:48,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:50,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:52,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:54,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:56,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:20:56,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:20:56,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:20:58,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:00,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:02,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:04,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:06,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:08,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:10,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:11,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:21:11,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:21:12,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:14,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:16,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:16,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:21:18,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:20,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:22,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:24,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:26,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:21:27,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:21:29,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:31,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:33,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:35,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:37,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:39,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:41,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:42,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:21:42,344 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:21:43,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:45,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:46,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:21:47,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:49,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:51,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:53,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:55,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:57,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:21:57,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:21:57,572 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:21:59,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:01,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:03,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:05,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:07,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:09,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:11,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:22:12,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:22:13,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:15,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:16,776 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:22:17,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:19,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:21,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:23,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:25,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:27,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:28,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:22:28,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:22:29,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:31,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:33,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:35,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:37,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:39,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:41,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:43,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:22:43,416 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:22:43,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:45,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:46,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:22:47,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:49,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:51,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:53,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:55,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:57,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:22:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:22:58,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:22:59,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:01,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:03,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:05,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:07,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:09,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:11,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:13,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:13,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:23:13,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:23:16,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:16,933 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:23:18,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:20,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:22,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:24,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:26,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:28,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:23:29,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:23:30,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:32,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:34,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:36,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:38,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:40,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:42,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:44,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:44,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:23:44,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:23:46,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:47,008 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:23:48,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:50,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:52,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:54,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:56,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:58,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:23:59,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:23:59,896 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:24:00,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:02,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:04,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:06,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:08,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:10,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:12,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:14,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:15,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:24:15,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:24:16,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:17,081 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:24:18,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:20,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:22,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:24,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:26,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:28,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:24:30,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:24:30,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:32,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:34,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:36,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:38,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:41,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:43,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:45,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:45,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:24:45,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:24:47,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:24:47,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:49,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:51,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:53,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:55,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:57,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:24:59,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:01,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:25:01,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:25:01,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:03,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:05,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:07,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:09,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:11,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:13,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:15,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:16,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:25:16,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:25:17,236 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:25:17,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:19,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:21,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:23,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:25,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:27,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:29,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:31,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:25:31,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:25:33,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:35,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:38,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:40,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:42,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:44,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:46,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:46,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:25:46,932 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:25:47,312 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:25:48,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:50,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:52,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:54,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:56,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:25:58,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:00,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:26:02,307 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:26:02,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:04,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:06,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:08,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:10,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:12,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:14,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:16,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:17,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:26:17,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:26:17,580 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:26:18,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:20,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:22,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:24,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:26,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:28,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:30,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:32,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:32,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:26:32,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:26:34,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:36,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:38,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:40,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:42,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:44,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:46,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:47,469 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:26:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:26:48,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:26:48,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:50,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:52,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:54,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:56,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:26:58,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:00,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:03,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:27:03,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:27:03,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:05,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:07,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:09,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:11,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:13,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:15,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:17,547 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:27:17,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:18,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:27:18,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:27:19,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:21,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:23,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:25,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:27,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:29,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:31,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:33,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:33,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:27:33,972 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:27:35,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:37,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:39,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:41,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:43,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:45,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:47,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:47,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:27:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:27:49,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:27:49,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:51,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:53,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:55,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:57,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:27:59,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:01,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:03,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:04,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:28:04,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:28:05,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:07,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:09,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:11,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:13,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:15,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:17,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:17,694 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:28:19,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:19,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:28:19,785 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:28:21,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:23,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:26,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:28,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:30,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:32,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:34,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:35,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:28:35,107 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:28:36,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:38,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:40,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:42,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:44,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:46,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:47,768 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:28:48,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:50,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:28:50,338 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:28:50,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:52,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:54,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:56,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:28:58,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:00,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:02,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:04,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:29:05,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:29:06,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:08,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:10,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:12,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:14,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:16,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:17,845 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:29:18,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:20,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:20,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:29:20,896 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:29:22,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:24,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:26,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:28,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:30,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:32,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:34,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:36,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:29:36,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:29:36,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:38,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:40,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:42,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:44,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:46,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:47,920 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:29:48,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:50,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:51,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:29:51,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:29:53,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:55,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:57,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:29:59,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:01,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:03,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:05,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:06,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:30:06,946 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:30:07,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:09,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:11,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:13,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:15,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:17,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:17,994 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:30:19,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:21,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:22,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:30:22,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:30:23,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:25,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:27,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:29,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:31,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:33,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:35,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:37,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:30:37,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:30:37,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:39,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:41,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:43,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:45,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:47,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:48,069 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:30:49,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:51,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:30:52,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:30:53,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:55,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:57,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:30:59,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:01,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:03,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:05,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:07,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:31:08,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:31:09,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:11,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:13,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:15,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:17,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:18,147 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:31:19,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:21,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:23,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:31:23,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:31:23,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:25,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:27,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:29,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:31,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:33,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:35,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:37,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:31:38,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:31:39,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:41,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:43,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:45,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:47,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:48,220 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:31:49,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:51,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:53,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:54,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:31:54,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:31:55,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:57,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:31:59,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:01,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:03,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:06,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:08,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:09,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:32:09,373 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:32:10,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:12,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:14,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:16,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:18,297 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:32:18,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:20,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:22,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:24,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:32:24,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:32:24,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:26,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:28,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:30,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:32,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:34,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:36,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:38,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:39,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:32:39,829 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:32:40,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:42,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:44,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:46,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:48,371 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:32:48,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:50,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:52,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:54,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:32:55,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:32:56,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:32:58,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:00,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:02,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:04,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:06,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:08,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:10,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:33:10,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:33:10,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:12,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:14,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:16,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:18,446 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:33:18,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:20,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:22,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:25,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:33:25,632 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:33:25,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:27,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:29,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:31,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:33,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:35,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:37,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:39,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:33:40,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:33:41,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:43,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:45,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:47,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:48,520 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:33:49,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:51,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:53,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:55,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:56,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:33:56,110 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:33:57,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:33:59,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:01,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:03,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:05,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:07,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:09,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:34:11,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:34:11,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:13,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:15,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:17,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:18,594 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:34:19,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:21,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:23,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:25,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:26,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:34:26,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:34:27,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:29,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:31,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:33,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:35,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:37,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:39,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:41,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:41,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:34:41,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:34:43,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:45,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:47,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:48,669 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:34:49,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:51,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:53,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:55,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:57,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:34:57,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:34:57,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:34:59,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:01,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:04,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:06,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:08,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:10,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:35:12,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:35:12,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:14,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:16,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:18,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:35:18,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:20,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:22,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:24,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:26,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:27,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:35:27,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:35:28,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:30,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:32,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:34,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:36,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:38,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:40,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:42,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:35:43,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:35:44,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:46,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:48,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:48,819 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:35:50,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:52,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:54,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:56,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:35:58,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:35:58,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:35:58,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:00,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:02,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:04,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:06,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:08,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:10,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:12,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:36:13,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:36:14,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:16,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:18,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:18,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:36:20,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:22,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:24,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:26,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:28,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:28,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:36:28,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:36:30,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:32,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:34,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:36,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:38,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:41,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:43,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:36:44,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:36:45,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:47,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:48,948 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:36:49,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:51,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:53,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:55,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:57,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:36:59,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:36:59,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:36:59,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:01,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:03,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:05,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:07,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:09,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:11,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:13,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:14,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:37:14,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:37:15,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:17,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:19,013 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:37:19,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:21,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:23,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:25,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:27,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:29,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:37:30,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:37:31,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:33,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:35,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:37,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:39,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:41,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:43,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:37:45,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:37:45,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:47,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:49,085 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:37:49,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:51,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:53,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:55,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:57,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:37:59,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:38:00,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:38:01,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:03,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:05,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:07,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:09,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:11,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:13,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:15,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:15,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:38:15,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:38:18,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:19,157 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:38:20,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:22,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:24,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:26,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:28,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:30,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:31,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:38:31,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:38:32,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:34,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:36,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:38,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:40,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:42,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:44,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:46,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:38:46,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:38:46,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:48,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:49,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:38:50,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:52,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:54,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:56,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:38:58,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:00,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:01,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:39:01,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:39:02,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:04,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:06,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:08,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:10,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:12,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:14,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:16,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:39:17,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:39:18,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:19,309 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:39:20,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:22,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:24,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:26,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:28,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:30,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:32,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:39:32,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:39:32,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:34,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:36,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:38,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:40,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:42,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:44,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:46,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:47,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:39:47,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:39:48,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:49,370 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:39:50,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:52,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:54,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:56,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:39:58,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:00,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:02,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:02,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:40:02,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:40:04,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:06,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:08,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:10,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:12,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:14,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:16,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:40:18,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:40:18,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:19,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:40:20,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:22,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:24,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:27,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:29,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:31,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:33,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:40:33,540 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:40:33,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:35,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:37,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:39,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:41,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:43,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:45,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:47,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:48,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:40:48,860 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:40:49,520 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:40:49,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:51,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:53,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:55,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:57,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:40:59,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:01,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:03,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:04,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:41:04,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:41:05,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:07,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:09,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:11,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:13,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:15,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:17,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:19,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:41:19,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:41:19,695 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:41:19,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:21,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:23,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:25,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:27,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:29,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:31,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:34,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:41:34,805 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:41:34,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:36,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:38,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:40,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:42,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:44,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:46,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:48,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:49,675 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:41:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:41:50,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:41:50,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:52,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:54,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:56,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:41:58,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:00,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:02,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:04,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:42:05,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:42:06,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:08,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:10,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:12,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:14,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:16,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:18,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:19,751 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:42:20,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:42:20,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:42:20,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:22,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:24,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:26,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:28,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:30,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:32,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:34,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:35,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:42:35,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:42:36,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:38,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:40,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:42,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:44,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:46,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:48,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:49,828 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:42:50,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:42:51,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:42:52,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:54,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:57,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:42:59,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:01,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:03,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:05,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:06,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:43:06,530 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:43:07,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:09,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:11,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:13,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:15,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:17,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:19,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:19,905 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:43:21,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:21,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:43:21,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:43:23,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:25,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:28,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:30,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:32,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:34,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:36,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:43:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:43:37,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:43:49,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:43:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:43:52,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:44:07,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:44:07,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:44:20,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:44:22,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:44:22,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:44:38,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:44:38,091 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:44:38,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:44:38,377 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 08:44:38,431 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 08:44:38,510 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 08:44:39,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 08:44:40,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:44:46,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:44:50,141 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:44:53,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:44:53,346 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:45:02,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:45:08,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:45:08,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:45:20,220 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:45:23,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:45:23,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:45:39,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:45:39,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:45:50,292 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:45:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:45:54,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:46:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:46:09,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:46:20,370 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:46:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:46:24,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:46:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:46:39,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:46:50,448 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:46:55,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:46:55,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:47:10,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:47:10,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:47:20,526 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:47:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:47:25,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:47:40,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:47:40,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:47:50,601 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:47:56,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:47:56,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:48:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:48:11,376 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:48:20,680 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:48:26,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:48:26,607 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:48:41,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:48:41,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:48:50,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:48:57,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:48:57,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:48:58,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:48:58,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:00,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:00,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:02,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:04,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:06,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:08,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:12,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:49:12,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:49:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:13,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:14,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:14,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:16,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:17,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:18,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:18,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:20,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:20,829 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:49:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:21,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:22,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:23,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:24,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:24,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:25,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:26,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:49:27,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:49:27,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:28,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:29,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:30,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:32,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:34,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:36,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:36,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:38,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:40,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:42,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:42,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:49:42,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:49:43,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:43,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:44,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:44,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:46,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:48,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:50,898 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:49:56,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:49:56,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:56,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:49:58,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:49:58,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:50:02,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:02,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:02,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:03,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:06,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:10,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:10,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:13,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:50:13,491 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:50:14,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:15,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:19,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:20,969 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:50:21,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:25,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:28,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:50:28,713 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:50:28,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:28,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:29,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:30,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:30,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:31,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:34,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:35,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:36,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:37,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:37,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:39,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:40,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:40,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:43,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:43,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:50:43,945 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:50:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:44,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:45,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:45,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:46,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:47,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:48,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:50,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:51,038 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:50:51,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:53,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:53,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:55,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:55,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:55,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:57,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:59,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:50:59,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:50:59,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:50:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:50:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:01,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:03,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:03,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:05,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:07,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:09,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:09,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:09,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:11,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:13,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:14,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:51:14,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:51:14,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:14,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:15,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:15,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:17,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:17,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:18,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:19,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:20,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:51:21,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:21,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:22,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:23,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:24,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:25,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:25,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:26,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:27,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:29,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:51:29,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:51:31,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:33,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:35,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:35,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:35,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:37,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:38,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:38,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:39,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:39,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:40,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:41,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:43,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:51:44,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:51:51,180 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:51:51,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:51:52,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:52,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:51:57,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:00,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:52:00,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:52:01,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:02,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:03,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:04,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:05,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:09,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:10,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:10,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:11,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:13,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:15,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:52:15,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:52:16,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:17,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:18,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:20,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:21,252 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:52:24,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:27,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:27,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:28,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:30,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:30,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:52:30,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:52:32,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:36,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:37,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:38,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:38,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:40,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:41,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:42,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:42,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:43,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:44,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:45,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:52:45,978 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:52:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:46,915 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 08:52:47,121 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 08:52:47,191 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 08:52:47,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 08:52:48,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:49,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:50,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:51,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:51,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:52:54,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:54,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:54,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:55,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:52:56,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:52:58,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:00,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:01,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:53:01,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:53:01,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:01,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:02,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:04,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:06,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:07,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:07,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:08,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:09,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:10,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:10,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:12,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:14,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:14,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:16,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:16,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:53:16,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:53:16,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:18,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:18,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:19,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:20,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:53:21,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:22,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:23,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:24,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:24,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:26,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:28,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:30,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:31,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:53:31,816 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:53:32,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:34,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:38,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:46,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:47,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:53:47,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:53:50,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:51,552 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:53:54,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:53:54,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:54,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:53:58,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:00,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:00,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:02,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:54:02,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:54:05,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:05,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:06,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:08,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:13,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:13,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:15,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:54:17,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:54:19,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:21,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:21,626 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:54:25,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:27,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:29,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:31,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:33,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:54:33,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:54:33,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:35,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:37,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:37,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:37,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:39,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:41,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:41,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:45,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:46,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:47,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:48,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:54:48,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:54:49,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:49,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:51,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:51,700 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:54:53,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:55,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:55,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:57,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:54:58,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:54:59,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:00,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:01,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:03,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:03,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:55:03,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:55:04,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:04,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:05,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:07,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:07,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:07,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:09,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:10,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:11,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:12,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:12,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:13,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:14,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:14,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:15,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:16,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:17,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:18,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:18,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:55:19,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:55:19,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:19,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:19,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:21,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:21,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:55:23,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:25,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:27,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:27,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:29,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:55:34,248 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:55:34,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:34,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:35,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:37,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:41,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:46,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:49,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:55:49,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:55:50,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:51,840 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:55:54,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:55:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:55,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:55:58,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:00,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:02,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:02,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:04,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:04,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:56:04,819 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:56:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:05,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:08,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:08,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:10,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:14,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:16,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:16,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:20,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:56:20,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:56:20,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:21,914 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:56:22,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:24,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:24,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:27,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:28,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:30,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:30,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:30,806 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 08:56:31,025 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 08:56:31,086 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 08:56:31,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 08:56:32,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:34,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:34,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:35,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:56:35,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:56:36,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:36,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:38,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:40,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:40,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:41,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:42,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:43,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:44,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:46,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:48,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:49,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:49,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:50,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:56:50,579 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:56:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:50,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:51,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,037 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:56:52,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:52,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:52,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:54,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:55,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:56,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:57,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:57,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:56:58,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:56:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:00,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:01,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:01,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:02,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:02,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:03,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:04,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:05,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:57:05,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:57:06,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:06,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:06,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:07,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:08,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:08,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:11,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:12,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:13,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:14,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:16,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:18,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:20,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:21,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:57:21,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:57:22,108 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:57:22,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:32,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:36,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:57:36,679 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:57:38,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:40,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:44,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:46,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:48,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:51,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:57:51,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:57:52,175 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:57:52,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:56,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:57:56,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:57:58,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:01,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:01,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:02,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:04,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:04,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:58:07,157 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:58:07,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:10,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:14,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:14,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:16,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:18,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:19,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:22,244 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:58:22,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:58:22,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:58:24,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:24,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:24,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:26,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:28,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:30,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:32,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:32,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:33,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:34,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:37,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:37,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:58:37,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:58:39,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:39,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:40,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:41,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:43,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:45,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:47,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:48,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:48,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:51,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:51,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:52,314 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:58:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:58:53,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:58:53,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:53,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:55,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:56,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:56,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:57,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:58,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:58,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:58:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:58:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:00,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:01,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:01,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:02,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:03,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:04,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:05,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:07,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:08,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:59:08,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:59:08,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:09,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:11,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:13,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:15,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:17,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:22,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:59:23,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:59:23,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:59:25,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:31,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:35,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:59:38,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:59:39,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:40,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:40,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:43,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:47,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:49,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:49,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:49,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:52,457 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 08:59:53,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:53,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 08:59:53,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 08:59:57,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 08:59:58,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 08:59:59,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:03,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:05,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:06,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:06,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:00:09,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:00:09,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:09,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:09,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:10,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:11,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:13,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:13,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:14,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:17,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:18,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:20,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:20,236 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 09:00:20,448 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 09:00:20,508 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 09:00:21,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 09:00:21,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:22,540 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:00:23,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:00:24,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:00:25,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:26,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:29,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:31,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:31,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:33,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:35,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:35,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:37,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:38,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:38,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:39,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:39,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:00:39,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:00:41,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:43,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:45,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:46,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:46,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:47,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:49,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:49,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:51,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:52,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:52,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:00:54,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:54,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:54,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:00:55,208 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:00:55,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:55,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:56,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:57,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:57,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:58,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:00:59,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:00:59,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:00,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:01,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:02,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:03,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:03,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:04,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:06,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:06,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:08,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:08,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:08,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:10,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:01:10,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:01:10,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:10,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:12,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:14,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:22,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:22,674 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:01:25,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:01:25,828 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:01:26,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:30,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:34,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:38,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:41,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:01:41,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:01:42,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:46,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:47,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:47,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:50,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:51,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:01:52,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:52,742 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:01:56,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:01:56,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:01:56,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:01:58,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:02,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:03,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:03,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:04,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:08,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:10,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:10,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:11,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:11,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:02:11,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:02:12,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:14,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:14,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:16,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:16,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:16,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:16,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:17,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:17,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:18,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:19,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:19,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:20,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:21,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:21,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:22,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:22,813 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:02:26,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:26,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:27,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:02:27,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:02:28,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:28,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:30,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:30,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:32,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:34,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:34,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:36,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:37,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:37,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:38,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:39,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:39,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:40,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:02:42,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:02:42,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:45,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:47,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:48,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:48,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:49,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:51,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:52,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:02:53,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:53,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:53,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:55,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:56,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:56,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:57,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:02:57,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:02:57,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:02:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:02:59,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:00,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:01,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:02,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:02,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:03,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:05,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:07,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:07,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:07,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:08,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:09,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:11,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:12,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:03:12,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:03:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:17,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:21,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:22,949 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:03:25,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:27,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:28,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:03:28,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:03:29,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:33,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:37,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:41,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:03:43,627 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:03:45,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:49,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:50,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:50,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:51,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:53,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:03:55,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:56,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:56,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:57,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:03:58,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:03:58,899 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:03:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:03:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:01,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:03,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:05,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:06,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:09,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:11,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:11,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:13,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:13,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:13,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:04:14,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:04:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,413 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 09:04:14,478 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 09:04:14,535 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 09:04:14,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 09:04:14,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:14,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:15,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:16,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:17,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:18,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:18,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:19,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:23,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:04:23,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:25,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:25,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:25,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:27,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:29,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:04:29,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:04:29,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:30,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:31,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:33,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:35,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:36,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:36,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:37,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:38,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:39,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:41,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:42,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:43,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:04:44,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:04:45,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:45,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:47,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:48,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:49,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:51,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:52,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:52,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:53,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:04:53,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:54,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:54,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:55,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:56,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:57,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:58,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:04:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:04:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:05:00,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:05:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:01,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:03,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:05,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:15,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:05:15,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:05:15,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:15,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:20,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:21,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:23,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:05:23,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:23,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:25,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:29,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:30,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:05:30,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:05:33,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:37,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:41,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:43,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:45,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:05:45,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:05:47,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:52,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:53,339 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:05:53,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:53,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:54,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:56,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:58,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:05:59,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:05:59,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:00,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:06:01,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:06:02,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:02,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:06,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:08,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:10,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:10,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:12,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:14,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:14,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:06:16,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:06:16,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:17,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:18,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:20,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:22,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:22,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,415 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:06:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:23,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:24,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:26,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:27,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:28,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:29,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:30,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:31,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:06:31,828 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:06:32,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:34,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:36,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:36,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:38,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:39,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:39,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:40,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:40,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:41,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:42,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:44,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:44,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:46,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:46,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:46,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:06:47,137 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:06:47,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:48,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:48,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:50,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:52,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:52,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:52,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:53,493 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:06:54,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:54,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:55,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:56,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:06:58,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:58,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:06:59,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:00,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:07:02,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:07:02,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:04,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:13,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:14,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:17,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:07:17,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:07:18,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:22,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:23,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:07:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:26,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:27,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:30,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:33,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:07:33,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:07:34,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:34,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:34,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:36,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:40,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:42,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:46,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:48,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:07:48,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:07:48,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:50,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:50,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:52,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:53,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:07:54,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:56,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:07:57,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:07:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:00,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:02,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:03,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:08:03,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:08:04,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:04,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:04,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:05,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:08:06,515 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 09:08:06,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:06,730 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 09:08:06,785 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 09:08:07,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 09:08:07,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:08,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:10,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:13,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:15,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:17,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:18,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:08:18,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:08:19,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:21,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:23,727 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:08:23,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:25,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:27,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:29,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:31,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:33,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:34,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:08:34,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:08:35,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:37,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:39,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:41,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:43,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:45,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:47,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:49,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:08:49,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:08:49,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:51,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:53,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:53,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:08:55,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:57,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:08:59,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:01,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:03,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:04,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:09:04,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:09:05,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:07,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:09,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:11,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:13,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:15,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:17,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:19,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:20,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:09:20,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:09:21,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:23,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:23,877 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:09:25,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:27,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:29,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:31,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:33,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:35,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:09:35,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:09:35,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:38,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:40,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:42,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:44,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:46,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:48,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:50,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:09:50,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:09:52,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:53,953 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:09:54,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:56,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:09:58,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:00,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:02,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:04,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:06,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:10:06,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:10:06,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:08,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:10,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:12,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:14,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:16,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:18,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:20,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:21,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:10:21,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:10:22,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:24,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:10:24,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:26,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:28,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:30,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:32,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:34,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:10:36,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:10:36,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:38,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:40,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:43,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:45,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:47,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:49,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:51,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:10:52,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:10:53,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:54,100 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:10:55,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:57,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:10:59,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:01,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:03,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:05,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:07,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:11:07,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:11:07,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:09,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:11,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:13,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:15,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:17,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:19,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:21,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:11:22,875 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:11:23,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:24,174 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:11:25,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:27,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:29,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:31,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:33,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:35,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:37,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:11:38,106 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:11:39,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:41,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:43,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:45,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:47,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:49,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:51,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:53,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:11:53,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:11:53,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:54,249 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:11:55,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:57,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:11:59,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:01,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:03,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:05,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:08,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:12:08,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:12:08,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:10,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:12,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:14,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:16,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:18,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:20,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:22,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:23,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:12:23,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:12:24,327 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:12:24,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:26,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:28,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:30,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:32,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:34,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:36,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:38,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:39,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:12:39,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:12:40,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:42,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:44,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:46,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:48,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:50,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:52,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:54,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:12:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:12:54,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:12:54,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:56,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:12:58,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:00,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:02,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:04,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:06,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:08,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:09,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:13:09,889 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:13:10,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:12,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:14,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:16,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:18,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:20,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:22,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:24,481 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:13:24,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:25,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:13:25,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:13:26,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:28,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:30,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:32,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:34,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:36,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:38,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:40,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:13:40,497 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:13:40,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:42,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:44,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:46,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:48,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:50,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:52,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:54,559 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:13:55,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:13:55,785 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:13:55,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:57,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:13:59,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:01,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:03,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:05,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:07,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:09,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:11,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:14:11,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:14:11,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:13,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:15,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:17,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:19,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:21,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:23,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:24,637 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:14:25,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:26,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:14:26,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:14:27,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:29,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:31,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:33,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:35,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:37,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:39,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:41,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:14:41,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:14:41,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:43,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:45,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:47,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:49,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:51,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:53,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:54,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:14:55,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:14:56,953 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:14:57,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:14:59,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:01,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:03,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:05,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:07,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:09,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:11,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:15:12,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:15:13,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:15,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:17,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:19,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:21,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:23,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:24,779 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:15:25,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:15:27,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:15:27,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:29,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:31,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:33,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:35,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:37,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:39,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:41,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:42,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:15:42,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:15:43,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:45,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:47,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:49,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:52,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:54,852 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:15:54,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:56,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:15:58,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:15:58,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:15:58,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:00,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:02,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:04,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:06,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:08,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:10,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:12,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:16:13,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:16:14,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:16,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:18,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:20,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:22,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:24,923 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:16:24,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:26,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:28,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:16:28,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:16:28,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:30,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:32,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:34,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:36,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:38,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:40,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:42,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:43,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:16:43,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:16:44,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:46,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:48,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:50,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:52,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:54,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:55,000 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:16:56,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:58,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:16:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:16:59,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:17:00,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:02,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:04,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:06,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:08,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:10,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:12,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:17:14,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:17:14,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:16,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:18,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:20,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:22,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:24,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:25,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:17:26,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:29,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:17:29,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:17:29,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:31,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:33,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:35,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:37,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:39,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:41,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:43,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:17:45,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:17:45,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:47,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:49,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:51,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:53,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:55,147 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:17:55,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:57,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:17:59,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:18:00,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:18:01,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:03,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:05,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:07,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:09,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:12,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:14,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:18:15,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:18:16,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:18,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:20,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:22,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:24,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:25,222 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:18:26,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:28,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:30,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:30,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:18:30,972 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:18:32,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:34,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:36,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:38,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:40,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:42,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:44,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:46,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:18:46,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:18:48,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:50,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:52,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:54,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:55,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:18:56,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:18:58,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:00,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:01,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:19:01,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:19:02,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:04,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:06,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:08,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:10,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:12,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:14,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:16,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:16,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:19:16,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:19:18,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:20,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:22,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:24,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:25,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:19:26,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:28,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:30,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:32,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:19:32,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:19:33,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:35,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:37,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:39,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:41,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:43,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:45,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:47,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:47,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:19:47,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:19:49,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:51,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:53,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:55,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:55,429 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:19:57,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:19:59,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:01,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:02,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:20:02,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:20:03,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:05,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:07,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:09,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:11,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:13,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:15,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:17,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:18,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:20:18,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:20:19,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:21,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:23,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:25,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:25,504 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:20:27,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:29,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:31,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:33,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:20:33,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:20:35,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:37,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:39,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:41,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:43,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:45,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:47,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:48,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:20:48,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:20:49,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:51,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:53,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:55,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:55,572 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:20:57,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:20:59,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:01,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:03,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:03,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:21:03,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:21:05,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:07,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:09,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:11,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:14,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:16,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:18,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:19,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:21:19,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:21:20,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:22,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:24,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:25,648 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:21:26,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:28,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:30,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:32,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:34,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:34,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:21:34,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:21:36,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:38,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:40,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:42,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:44,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:46,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:48,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:49,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:21:49,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:21:50,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:52,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:54,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:55,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:21:56,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:21:58,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:00,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:02,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:04,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:04,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:22:04,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:22:06,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:08,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:10,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:12,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:14,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:16,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:18,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:20,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:20,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:22:20,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:22:22,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:24,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:25,789 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:22:26,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:28,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:30,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:32,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:34,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:35,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:22:35,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:22:36,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:38,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:40,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:42,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:44,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:46,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:48,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:50,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:22:50,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:22:52,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:54,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:55,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:22:56,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:22:58,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:00,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:03,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:05,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:05,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:23:05,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:23:07,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:09,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:11,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:13,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:15,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:17,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:19,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:21,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:21,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:23:21,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:23:23,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:25,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:25,942 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:23:27,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:29,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:31,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:33,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:35,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:23:36,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:23:37,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:39,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:41,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:43,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:45,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:47,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:49,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:51,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:51,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:23:51,959 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:23:53,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:55,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:56,017 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:23:57,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:23:59,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:01,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:03,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:05,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:07,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:24:07,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:24:09,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:11,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:14,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:16,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:18,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:20,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:22,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:22,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:24:22,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:24:24,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:26,090 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:24:26,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:28,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:30,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:32,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:34,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:36,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:38,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:24:38,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:24:38,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:40,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:42,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:44,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:46,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:48,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:50,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:52,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:53,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:24:53,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:24:54,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:56,153 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:24:56,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:24:58,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:00,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:02,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:04,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:06,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:08,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:08,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:25:08,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:25:10,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:12,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:14,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:16,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:18,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:20,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:22,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:25:24,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:25:24,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:26,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:26,228 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:25:28,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:30,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:32,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:34,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:36,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:38,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:25:39,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:25:40,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:42,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:45,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:47,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:49,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:51,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:53,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:54,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:25:54,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:25:55,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:56,303 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:25:57,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:25:59,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:01,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:03,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:05,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:07,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:09,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:26:09,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:26:11,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:13,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:15,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:17,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:19,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:21,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:23,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:26:25,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:26:25,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:26,360 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:26:27,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:29,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:31,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:33,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:35,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:37,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:39,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:40,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:26:40,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:26:41,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:43,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:45,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:47,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:49,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:51,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:53,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:55,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:55,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:26:55,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:26:56,435 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:26:57,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:26:59,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:01,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:03,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:05,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:08,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:10,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:10,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:27:10,971 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:27:12,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:14,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:16,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:18,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:20,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:22,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:24,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:26,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:27:26,282 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:27:26,507 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:27:28,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:30,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:32,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:34,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:36,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:38,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:40,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:41,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:27:41,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:27:42,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:44,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:46,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:48,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:50,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:52,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:54,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:56,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:27:56,575 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:27:56,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:27:56,878 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:27:58,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:00,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:02,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:04,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:06,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:08,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:10,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:12,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:28:12,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:28:12,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:14,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:16,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:18,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:20,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:22,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:24,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:26,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:26,651 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:28:27,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:28:27,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:28:28,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:30,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:32,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:34,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:36,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:38,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:40,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:42,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:42,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:28:42,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:28:44,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:46,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:48,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:50,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:52,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:54,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:56,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:28:56,732 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:28:57,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:28:57,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:28:58,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:00,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:02,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:04,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:07,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:09,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:11,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:13,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:29:13,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:29:13,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:15,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:17,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:19,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:21,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:23,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:25,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:26,812 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:29:27,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:28,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:29:28,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:29:29,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:31,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:33,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:35,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:37,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:39,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:41,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:43,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:29:43,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:29:45,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:47,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:49,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:51,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:53,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:55,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:56,888 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:29:57,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:29:58,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:29:58,889 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:29:59,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:01,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:03,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:05,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:07,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:09,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:11,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:13,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:30:14,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:30:15,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:17,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:19,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:21,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:23,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:25,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:26,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:30:27,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:29,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:30:29,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:30:32,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:34,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:36,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:38,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:40,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:42,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:44,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:44,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:30:44,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:30:46,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:48,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:50,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:52,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:54,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:56,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:30:57,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:30:58,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:31:00,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:31:00,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:02,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:04,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:06,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:08,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:10,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:12,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:14,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:31:15,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:31:16,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:18,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:20,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:22,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:24,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:26,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:27,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:31:28,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:30,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:30,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:31:30,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:31:32,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:34,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:36,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:38,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:40,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:43,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:45,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:31:45,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:31:47,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:49,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:51,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:53,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:55,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:57,191 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:31:57,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:31:59,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:32:01,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:32:01,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:03,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:05,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:07,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:09,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:11,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:13,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:15,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:16,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:32:16,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:32:17,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:19,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:21,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:23,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:25,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:27,266 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:32:27,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:29,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:31,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:32:31,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:32:33,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:35,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:37,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:39,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:41,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:43,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:45,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:47,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:32:47,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:32:47,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:49,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:51,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:53,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:55,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:57,346 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:32:57,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:32:59,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:01,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:33:02,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:33:04,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:06,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:08,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:10,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:12,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:14,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:16,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:17,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:33:17,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:33:18,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:20,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:22,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:24,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:26,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:27,420 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:33:28,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:30,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:32,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:32,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:33:32,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:33:34,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:36,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:38,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:40,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:42,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:44,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:46,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:48,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:33:48,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:33:48,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:50,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:52,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:54,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:56,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:33:57,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:33:58,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:00,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:02,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:03,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:34:03,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:34:04,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:06,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:08,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:10,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:12,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:14,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:16,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:18,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:18,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:34:18,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:34:20,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:22,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:25,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:27,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:27,573 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:34:29,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:31,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:33,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:34,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:34:34,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:34:35,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:37,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:39,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:41,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:43,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:45,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:47,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:49,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:34:49,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:34:51,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:53,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:55,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:57,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:34:57,649 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:34:59,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:01,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:03,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:35:05,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:35:05,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:07,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:09,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:11,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:13,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:15,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:17,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:19,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:35:20,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:35:21,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:23,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:25,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:27,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:27,723 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:35:29,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:31,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:33,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:35,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:35:35,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:35:37,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:39,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:41,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:43,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:45,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:47,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:49,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:51,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:35:51,103 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:35:52,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:54,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:56,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:35:57,799 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:35:58,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:00,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:02,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:04,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:06,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:36:06,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:36:06,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:08,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:10,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:12,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:14,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:16,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:18,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:20,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:21,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:36:21,681 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:36:22,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:24,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:26,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:27,875 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:36:28,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:30,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:32,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:34,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:36,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:36,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:36:36,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:36:38,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:40,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:42,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:44,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:46,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:48,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:50,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:36:52,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:36:52,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:54,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:56,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:36:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:36:58,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:00,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:02,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:04,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:06,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:07,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:37:07,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:37:08,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:10,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:12,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:14,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:16,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:18,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:20,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:22,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:23,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:37:23,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:37:24,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:26,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:28,023 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:37:28,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:30,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:32,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:34,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:36,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:38,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:37:38,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:37:40,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:43,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:45,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:47,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:49,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:51,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:53,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:37:53,852 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:37:55,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:57,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:37:58,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:37:59,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:01,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:03,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:05,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:07,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:09,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:38:09,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:38:09,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:11,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:13,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:15,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:17,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:19,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:21,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:23,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:24,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:38:24,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:38:25,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:27,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:28,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:38:29,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:31,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:33,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:35,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:37,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:39,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:39,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:38:39,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:38:41,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:43,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:45,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:47,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:49,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:51,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:53,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:55,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:38:55,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:38:55,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:57,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:38:58,241 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:38:59,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:01,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:03,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:05,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:07,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:09,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:10,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:39:10,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:39:11,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:13,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:15,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:17,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:19,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:21,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:23,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:25,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:25,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:39:25,786 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:39:27,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:28,312 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:39:29,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:32,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:34,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:36,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:38,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:40,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:41,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:39:41,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:39:42,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:44,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:46,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:48,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:50,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:52,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:54,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:56,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:39:56,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:39:56,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:39:58,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:39:58,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:00,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:02,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:04,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:06,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:08,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:10,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:11,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:40:11,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:40:12,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:14,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:16,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:18,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:20,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:22,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:24,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:26,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:40:26,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:40:28,465 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:40:28,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:30,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:32,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:34,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:36,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:38,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:40,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:40:42,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:40:42,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:44,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:47,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:49,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:51,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:53,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:55,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:57,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:40:57,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:40:57,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:40:58,543 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:40:59,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:01,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:03,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:05,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:07,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:09,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:11,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:41:12,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:41:13,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:15,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:17,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:19,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:21,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:23,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:25,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:27,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:27,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:41:27,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:41:28,622 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:41:29,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:31,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:33,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:35,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:37,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:39,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:41,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:43,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:41:43,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:41:43,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:45,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:47,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:49,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:51,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:53,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:55,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:57,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:41:58,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:41:58,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:41:58,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:41:59,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:01,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:03,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:05,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:07,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:09,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:11,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:13,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:13,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:42:13,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:42:15,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:17,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:19,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:21,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:23,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:25,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:27,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:28,777 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:42:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:42:29,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:42:29,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:32,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:34,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:36,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:38,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:40,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:42,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:44,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:42:44,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:42:44,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:46,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:48,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:50,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:52,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:54,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:56,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:58,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:42:58,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:42:59,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:42:59,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:43:00,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:02,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:04,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:06,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:08,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:10,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:12,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:14,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:15,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:43:15,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:43:16,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:18,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:20,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:22,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:24,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:26,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:28,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:28,926 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:43:30,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:43:30,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:43:30,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:32,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:34,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:36,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:38,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:40,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:42,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:44,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:45,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:43:45,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:43:46,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:48,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:50,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:52,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:54,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:57,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:43:59,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:43:59,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:00,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:44:00,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:44:01,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:03,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:05,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:07,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:09,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:11,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:13,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:15,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:44:16,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:44:17,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:19,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:21,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:23,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:25,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:27,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:29,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:44:29,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:31,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:44:31,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:44:31,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:33,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:35,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:37,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:39,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:41,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:43,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:45,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:46,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:44:46,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:44:47,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:49,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:51,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:53,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:55,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:57,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:44:59,150 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:44:59,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:01,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:02,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:45:02,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:45:03,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:05,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:07,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:09,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:11,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:13,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:15,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:17,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:45:17,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:45:17,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:19,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:21,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:23,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:25,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:27,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:29,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:45:29,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:31,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:32,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:45:32,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:45:33,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:35,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:37,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:39,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:41,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:43,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:45,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:47,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:45:47,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:45:48,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:50,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:52,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:54,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:56,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:58,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:45:59,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:46:00,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:02,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:03,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:46:03,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:46:04,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:06,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:08,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:10,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:12,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:14,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:16,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:18,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:46:18,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:46:18,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:20,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:22,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:24,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:26,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:28,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:29,365 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:46:30,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:32,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:33,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:46:33,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:46:34,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:36,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:38,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:40,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:42,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:44,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:46,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:48,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:49,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:46:49,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:46:50,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:52,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:55,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:57,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:46:59,441 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:46:59,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:01,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:03,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:47:04,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:47:05,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:07,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:09,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:11,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:13,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:15,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:17,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:19,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:19,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:47:19,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:47:21,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:23,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:25,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:27,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:29,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:47:29,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:31,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:33,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:35,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:47:35,126 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:47:35,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:37,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:39,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:41,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:43,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:45,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:47,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:49,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:50,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:47:50,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:47:51,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:53,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:55,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:57,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:47:59,593 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:47:59,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:01,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:03,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:05,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:48:05,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:48:05,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:07,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:09,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:11,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:13,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:15,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:17,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:19,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:48:20,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:48:21,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:23,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:25,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:27,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:29,671 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:48:30,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:32,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:34,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:36,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:48:36,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:48:36,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:38,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:40,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:42,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:44,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:46,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:48,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:50,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:51,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:48:51,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:48:52,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:54,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:56,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:58,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:48:59,748 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:49:00,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:02,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:04,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:06,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:06,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:49:06,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:49:08,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:10,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:12,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:14,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:16,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:18,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:20,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:22,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:49:22,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:49:22,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:24,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:26,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:28,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:29,827 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:49:30,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:32,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:34,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:36,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:37,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:49:37,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:49:38,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:40,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:42,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:44,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:46,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:48,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:50,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:52,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:49:52,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:49:52,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:54,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:56,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:58,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:49:59,900 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:50:00,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:50:02,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:50:05,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:50:07,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:50:07,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:50:07,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:50:09,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:50:23,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:50:23,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:50:29,975 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:50:38,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:50:38,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:50:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:50:53,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:51:00,038 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:51:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:51:08,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:51:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:51:09,170 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 09:51:09,170 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 09:51:09,278 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 09:51:09,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 09:51:10,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:51:11,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:51:15,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:51:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:51:24,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:51:30,130 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:51:31,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:51:39,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:51:39,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:51:54,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:51:54,845 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:52:00,204 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:52:10,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:52:10,077 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:52:25,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:52:25,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:52:30,278 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:52:40,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:52:40,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:52:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:52:55,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:53:00,352 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:53:11,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:53:11,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:53:26,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:53:26,255 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:53:30,429 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:53:41,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:53:41,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:53:56,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:53:56,826 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:54:00,507 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:54:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:54:12,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:54:27,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:54:27,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:54:30,582 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:54:42,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:54:42,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:54:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:54:57,758 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:55:00,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:55:12,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:55:12,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:55:25,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:55:28,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:55:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:28,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:29,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:30,736 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:55:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:31,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:33,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:33,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:33,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:34,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:35,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:37,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:37,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:39,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:41,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:42,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:42,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:43,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:55:43,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:55:43,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:45,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:46,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:46,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:47,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:48,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:49,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:51,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:52,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:53,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:55,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:57,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:57,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:55:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:55:58,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:55:58,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:55:59,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:00,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:00,810 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:56:01,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:03,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:04,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:04,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:05,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:08,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:09,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:10,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:11,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:12,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:12,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:13,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:56:14,002 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:56:14,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:14,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:15,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:16,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:18,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:26,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:29,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:56:29,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:56:30,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:30,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:56:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:32,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:36,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:40,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:41,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:41,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:42,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:44,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:56:44,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:56:44,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:47,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:48,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:53,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:55,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:57,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:57,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:56:59,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:56:59,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:56:59,786 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:57:00,945 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:57:01,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:05,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:07,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:09,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:09,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:09,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:11,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:11,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:13,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:15,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:15,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:57:15,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:57:17,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:17,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:20,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:21,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:23,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:25,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:26,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:26,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:27,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:29,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:57:30,373 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:57:30,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:30,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:31,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:31,017 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:57:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:34,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:35,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:36,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:37,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:39,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:41,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:43,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:43,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:45,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:57:45,788 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:57:45,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:45,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:47,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:49,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:50,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:50,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:51,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:55,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:56,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:57:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:57,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:58,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:57:59,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:01,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:01,087 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:58:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:58:01,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:58:03,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:03,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:03,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:05,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:05,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:06,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:07,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:08,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:09,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:11,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:13,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:15,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:15,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:16,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:58:16,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:58:17,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:17,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:25,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:27,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:29,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:31,152 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:58:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:58:31,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:58:32,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:36,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:40,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:44,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:44,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:58:47,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:58:48,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:50,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:51,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:58:54,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:58:56,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:00,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:01,222 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:59:02,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:02,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:59:02,422 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:59:04,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:04,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:06,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:08,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:09,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:10,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:11,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:14,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:15,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:16,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:16,777 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 09:59:17,001 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 09:59:17,074 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 09:59:17,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:59:17,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:59:18,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 09:59:18,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:20,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:24,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:25,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:26,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:26,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:28,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:30,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:31,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 09:59:32,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:33,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:59:33,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:59:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:34,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:34,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:36,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:38,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:38,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:40,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:42,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:44,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:45,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:45,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:46,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:47,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:48,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:48,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 09:59:48,261 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 09:59:49,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:49,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:54,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 09:59:56,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:56,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:57,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 09:59:58,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:00,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:01,370 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:00:02,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:03,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:00:03,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:00:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:03,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:04,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:06,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:06,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:08,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:10,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:10,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:11,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:00:18,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:00:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:19,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:21,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:23,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:24,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:25,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:29,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:29,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:31,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:00:32,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:32,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:33,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:33,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:00:33,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:00:36,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:37,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:43,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:44,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:47,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:48,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:48,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:49,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:00:49,323 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:00:51,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:51,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:52,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:00:53,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:57,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:00:59,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:01,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:01:03,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:04,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:01:04,600 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:01:05,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:07,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:07,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:07,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:09,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:11,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:13,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:15,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:17,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:19,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:19,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:01:19,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:01:21,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:23,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:25,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:26,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:27,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:28,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:29,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:29,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:31,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:31,594 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:01:33,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:34,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:34,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:01:35,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:01:35,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:35,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:35,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:37,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:38,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:38,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:39,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:39,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:40,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:41,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:43,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:43,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:45,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:46,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:46,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:47,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:49,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:01:50,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:01:50,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:50,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:51,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:51,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:53,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:53,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:54,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:01:55,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:55,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:01:58,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:00,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:01,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:01,661 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:02:02,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:02,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:03,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:04,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:02:05,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:02:06,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:08,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:11,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:14,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:20,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:20,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:02:20,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:02:21,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:24,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:25,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:28,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:29,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:31,737 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:02:32,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:34,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:35,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:02:35,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:02:36,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:36,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:36,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:38,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:42,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:44,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:46,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:48,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:49,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:49,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:51,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:02:51,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:02:52,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:53,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:53,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:02:54,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:02:56,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:00,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:01,812 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:03:02,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:04,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:06,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:03:06,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:03:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:07,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:08,608 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 10:03:08,824 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 10:03:08,897 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 10:03:09,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 10:03:10,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:11,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:14,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:16,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:18,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:21,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:03:21,785 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:03:22,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:22,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:24,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:26,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:26,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:30,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:31,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:31,957 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:03:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:32,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:33,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:35,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:35,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:03:37,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:03:37,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:37,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:39,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:41,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:43,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:44,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:45,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:47,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:48,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:48,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:49,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:49,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:50,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:51,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:52,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:03:52,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:03:52,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:52,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:53,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:53,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:55,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:57,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:03:58,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:58,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:03:59,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:01,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:02,029 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:04:07,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:04:07,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:04:11,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:15,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:15,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:17,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:17,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:19,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:22,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:04:22,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:04:23,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:23,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:23,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:24,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:27,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:28,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:28,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:31,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:32,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:04:32,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:32,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:35,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:37,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:38,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:04:38,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:04:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:41,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:43,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:47,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:49,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:49,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:51,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:53,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:04:53,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:04:53,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:53,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:54,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:55,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:04:57,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:57,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:04:59,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:01,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:02,167 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:05:03,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:03,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:03,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:05,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:07,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:08,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:05:08,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:05:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:11,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:13,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:15,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:17,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:17,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:19,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:21,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:22,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:22,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:23,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:23,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:05:23,869 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:05:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:24,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:25,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:27,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:29,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:31,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:31,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:32,233 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:05:33,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:33,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:35,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:36,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:37,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:38,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:38,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:39,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:05:39,223 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:05:40,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:42,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:42,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:42,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:43,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:44,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:44,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:45,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:46,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:46,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:47,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:48,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:50,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:51,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:51,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:52,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:53,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:53,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:05:54,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:05:54,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:05:56,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:05:58,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:02,307 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:06:08,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:09,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:06:09,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:06:12,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:15,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:18,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:20,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:22,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:24,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:25,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:06:25,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:06:26,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:26,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:28,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:31,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:32,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:32,373 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:06:33,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:34,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:36,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:36,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:38,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:39,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:40,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:06:40,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:06:42,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:43,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:43,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:44,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:48,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:50,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:52,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:53,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:53,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:55,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:06:55,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:06:56,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:57,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:57,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:06:58,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:06:58,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:00,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:01,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:01,836 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 10:07:02,032 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 10:07:02,105 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 10:07:02,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 10:07:02,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:02,506 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:07:04,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:06,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:08,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:09,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:09,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:10,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:11,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:07:11,095 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:07:12,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:12,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:14,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:16,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:16,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:18,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:20,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:23,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:23,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:25,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:26,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:07:26,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:07:27,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:27,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:29,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:30,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:30,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:31,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:32,578 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:07:33,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:34,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:34,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:35,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:37,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:39,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:39,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:41,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:07:41,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:07:42,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:42,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:43,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:45,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:45,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:46,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:46,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:47,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:47,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:48,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:49,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:49,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:07:51,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:53,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:55,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:07:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:07:56,794 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:08:01,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:02,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:02,652 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:08:07,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:11,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:12,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:08:12,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:08:15,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:15,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:16,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:16,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:18,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:19,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:20,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:20,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:22,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:23,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:25,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:27,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:08:27,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:08:29,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:29,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:32,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:08:33,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:33,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:33,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:35,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:36,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:38,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:39,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:40,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:41,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:42,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:08:42,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:08:43,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:47,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:47,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:49,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:50,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:50,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:51,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:55,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:55,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:55,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:57,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:08:57,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:08:58,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:08:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:08:59,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:00,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:02,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:02,792 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:09:04,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:08,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:10,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:11,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:11,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:12,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:13,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:09:13,063 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:09:14,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:15,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:16,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:18,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:20,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:21,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:21,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:22,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:24,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:25,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:27,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:28,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:28,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:09:28,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:09:30,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:31,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:31,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:32,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:32,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:09:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:33,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:34,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:35,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:36,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:37,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:38,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:40,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:40,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:40,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:41,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:42,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:42,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:09:43,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:09:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:44,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:46,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:48,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:54,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:09:56,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:09:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:09:58,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:10:02,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:02,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:10:07,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:07,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:07,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:11,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:10:14,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:10:15,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:18,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:19,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:19,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:20,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:21,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:23,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:23,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:25,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:29,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:29,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:10:29,406 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:10:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:33,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:33,009 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:10:33,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:35,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:35,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:36,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:38,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:39,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:40,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:41,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:42,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:42,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:43,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:44,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:10:44,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:10:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:47,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:49,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:50,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:51,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:51,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:51,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:52,349 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 10:10:52,550 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 10:10:52,620 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 10:10:53,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 10:10:53,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:55,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:57,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:10:58,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:58,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:10:59,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:00,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:11:00,003 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:11:00,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:00,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:01,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:03,094 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:11:03,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:05,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:07,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:09,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:11,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:13,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:14,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:15,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:11:15,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:11:15,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:15,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:17,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:18,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:19,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:21,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:23,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:25,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:26,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:26,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:27,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:29,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:30,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:11:30,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:11:31,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:31,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:11:33,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:34,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:35,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:36,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:36,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:37,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:39,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:39,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:40,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:41,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:43,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:45,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:46,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:11:46,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:11:51,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:11:52,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:11:57,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:01,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:12:01,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:12:01,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:03,236 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:12:05,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:10,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:12,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:14,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:16,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:16,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:12:16,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:12:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:17,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:18,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:20,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:22,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:22,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:26,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:26,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:28,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:30,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:30,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:31,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:12:31,853 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:12:32,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:32,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:33,311 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:12:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:34,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:36,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:36,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:37,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:38,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:39,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:39,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:40,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:41,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:42,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:44,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:44,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:45,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:45,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:12:47,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:12:47,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:47,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:48,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:49,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:49,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:50,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:52,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:54,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:56,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:57,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:12:58,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:12:58,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:00,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:00,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:02,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:13:02,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:13:02,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:03,382 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:13:04,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:06,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:08,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:09,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:09,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:10,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:12,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:13,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:14,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:16,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:17,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:13:17,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:13:18,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:20,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:21,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:21,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:22,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:23,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:23,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:24,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:26,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:26,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:27,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:27,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:28,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:30,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:30,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:32,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:32,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:13:32,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:13:33,449 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:13:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:33,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:34,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:36,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:38,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:39,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:46,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:47,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:47,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:13:48,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:13:52,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:54,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:56,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:13:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:13:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:00,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:03,523 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:14:03,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:14:03,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:14:04,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:05,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:08,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:10,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:12,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:12,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:14,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:16,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:18,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:14:18,853 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:14:21,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:23,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:24,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:24,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:27,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:27,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:29,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:30,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:31,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:33,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:33,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:14:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:14:34,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:14:35,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:35,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:36,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:36,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:37,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:39,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:39,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:40,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:40,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:41,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:14:41,126 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 10:14:41,338 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 10:14:41,409 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 10:14:41,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 10:14:41,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:43,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:45,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:47,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:49,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:14:49,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:14:49,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:51,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:53,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:55,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:57,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:14:59,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:01,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:03,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:03,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:15:04,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:15:04,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:15:05,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:07,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:09,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:11,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:13,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:15,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:17,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:19,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:19,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:15:19,890 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:15:21,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:23,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:25,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:27,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:29,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:31,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:33,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:33,759 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:15:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:15:35,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:15:35,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:38,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:40,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:42,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:44,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:46,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:48,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:50,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:50,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:15:50,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:15:52,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:54,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:56,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:15:58,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:00,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:02,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:03,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:16:04,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:05,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:16:05,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:16:06,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:08,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:10,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:12,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:14,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:16,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:18,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:20,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:21,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:16:21,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:16:22,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:24,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:26,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:28,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:30,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:32,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:33,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:16:34,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:36,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:16:36,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:16:36,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:38,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:40,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:42,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:44,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:46,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:49,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:51,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:51,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:16:51,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:16:53,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:55,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:57,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:16:59,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:01,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:03,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:03,992 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:17:05,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:07,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:17:07,029 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:17:07,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:09,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:11,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:13,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:15,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:17,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:19,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:21,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:22,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:17:22,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:17:23,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:25,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:27,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:29,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:31,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:33,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:34,067 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:17:35,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:37,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:37,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:17:37,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:17:39,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:41,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:43,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:45,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:47,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:49,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:51,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:53,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:17:53,040 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:17:53,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:55,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:57,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:17:59,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:01,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:03,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:04,145 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:18:05,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:07,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:18:08,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:18:09,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:11,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:13,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:15,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:17,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:19,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:21,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:23,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:23,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:18:23,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:18:25,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:27,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:29,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:31,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:34,218 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:18:34,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:36,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:38,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:38,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:18:38,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:18:40,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:42,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:44,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:46,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:48,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:50,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:52,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:18:54,138 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:18:54,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:56,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:18:58,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:00,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:02,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:04,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:19:04,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:06,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:08,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:09,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:19:09,363 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:19:10,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:12,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:14,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:16,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:18,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:20,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:22,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:24,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:19:24,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:19:26,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:28,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:30,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:32,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:34,369 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:19:34,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:36,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:38,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:19:39,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:19:40,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:42,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:44,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:46,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:48,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:50,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:52,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:54,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:55,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:19:55,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:19:57,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:19:59,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:01,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:03,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:04,443 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:20:05,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:07,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:09,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:10,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:20:10,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:20:11,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:13,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:15,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:17,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:19,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:21,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:23,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:25,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:25,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:20:25,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:20:27,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:29,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:31,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:33,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:34,515 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:20:35,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:37,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:39,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:20:41,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:20:41,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:43,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:45,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:47,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:49,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:51,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:53,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:55,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:56,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:20:56,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:20:57,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:20:59,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:01,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:03,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:04,576 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:21:05,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:07,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:09,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:11,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:21:11,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:21:11,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:13,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:15,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:17,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:19,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:21,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:23,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:25,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:26,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:21:26,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:21:27,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:29,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:31,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:33,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:34,650 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:21:35,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:37,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:39,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:41,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:42,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:21:42,338 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:21:43,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:45,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:47,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:49,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:51,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:53,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:55,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:57,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:21:57,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:21:57,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:21:59,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:02,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:04,712 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:22:04,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:06,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:08,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:10,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:12,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:22:12,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:22:14,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:16,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:18,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:20,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:22,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:24,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:26,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:22:28,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:22:28,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:30,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:32,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:34,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:34,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:22:36,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:38,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:40,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:42,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:43,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:22:43,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:22:44,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:46,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:48,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:50,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:52,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:54,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:56,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:22:58,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:22:58,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:22:58,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:00,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:02,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:04,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:04,860 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:23:06,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:08,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:10,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:13,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:14,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:23:14,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:23:15,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:17,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:19,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:21,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:23,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:25,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:27,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:29,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:23:29,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:23:29,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:31,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:33,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:34,935 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:23:35,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:37,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:39,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:41,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:43,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:44,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:23:44,604 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:23:45,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:47,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:49,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:51,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:53,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:55,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:57,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:59,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:23:59,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:23:59,878 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:24:01,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:03,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:05,009 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:24:05,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:07,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:09,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:11,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:13,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:24:15,212 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:24:15,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:17,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:19,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:21,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:23,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:25,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:27,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:29,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:30,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:24:30,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:24:31,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:33,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:35,085 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:24:35,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:37,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:39,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:41,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:43,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:45,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:45,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:24:45,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:24:47,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:50,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:52,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:54,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:56,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:24:58,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:00,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:25:01,162 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:25:02,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:04,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:05,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:25:06,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:08,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:10,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:12,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:14,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:16,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:25:16,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:25:16,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:18,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:20,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:22,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:24,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:26,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:28,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:30,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:31,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:25:31,805 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:25:32,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:34,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:35,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:25:36,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:38,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:40,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:42,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:44,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:46,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:47,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:25:47,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:25:48,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:50,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:52,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:54,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:56,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:25:58,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:00,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:02,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:26:02,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:26:02,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:04,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:05,305 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:26:06,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:08,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:10,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:12,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:14,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:16,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:17,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:26:17,664 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:26:18,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:20,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:23,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:25,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:27,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:29,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:31,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:32,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:26:32,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:26:33,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:35,378 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:26:35,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:37,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:39,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:41,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:43,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:45,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:47,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:48,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:26:48,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:26:49,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:51,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:53,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:55,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:57,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:26:59,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:01,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:03,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:27:03,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:27:03,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:05,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:27:05,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:07,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:09,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:11,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:13,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:15,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:17,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:18,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:27:18,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:27:19,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:21,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:23,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:25,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:27,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:29,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:31,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:33,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:34,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:27:34,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:27:35,517 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:27:35,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:37,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:39,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:41,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:43,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:45,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:47,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:49,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:27:49,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:27:49,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:51,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:53,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:55,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:57,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:27:59,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:01,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:03,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:04,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:28:04,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:28:05,583 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:28:05,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:07,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:09,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:11,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:13,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:16,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:18,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:19,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:28:19,993 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:28:20,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:22,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:24,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:26,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:28,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:30,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:32,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:34,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:35,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:28:35,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:28:35,786 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:28:36,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:38,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:40,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:42,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:44,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:46,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:48,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:28:50,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:28:50,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:52,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:54,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:56,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:28:58,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:00,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:02,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:04,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:05,732 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:29:06,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:29:06,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:29:06,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:08,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:10,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:12,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:14,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:16,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:18,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:20,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:21,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:29:21,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:29:22,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:24,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:26,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:28,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:30,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:32,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:34,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:35,808 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:29:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:29:36,864 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:29:36,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:38,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:40,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:42,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:45,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:47,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:49,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:51,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:29:52,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:29:53,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:55,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:57,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:29:59,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:01,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:03,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:05,882 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:30:05,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:30:07,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:30:07,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:09,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:11,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:13,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:15,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:17,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:19,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:21,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:22,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:30:22,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:30:23,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:25,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:27,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:29,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:31,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:33,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:35,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:30:35,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:37,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:38,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:30:38,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:30:39,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:41,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:43,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:45,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:47,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:49,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:51,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:53,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:54,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:30:54,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:30:55,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:57,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:30:59,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:01,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:03,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:05,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:06,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:31:07,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:09,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:10,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:31:10,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:31:11,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:13,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:15,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:17,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:19,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:21,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:23,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:31:25,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:31:25,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:27,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:29,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:31,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:34,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:36,113 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:31:36,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:38,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:40,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:31:40,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:31:40,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:42,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:44,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:46,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:48,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:50,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:52,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:54,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:56,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:31:56,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:31:56,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:31:59,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:01,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:03,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:05,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:06,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:32:07,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:09,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:11,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:32:11,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:32:13,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:15,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:17,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:19,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:21,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:23,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:25,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:32:26,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:32:27,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:29,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:31,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:33,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:35,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:36,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:32:37,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:39,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:41,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:42,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:32:42,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:32:43,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:45,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:47,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:49,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:51,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:53,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:55,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:57,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:32:57,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:32:57,414 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:32:59,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:01,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:03,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:06,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:06,337 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:33:08,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:10,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:12,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:33:12,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:33:14,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:16,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:18,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:20,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:22,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:24,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:26,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:28,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:33:28,057 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:33:30,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:32,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:34,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:36,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:36,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:33:38,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:40,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:42,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:33:43,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:33:44,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:46,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:48,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:50,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:52,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:54,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:56,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:58,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:33:58,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:33:58,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:34:00,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:02,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:04,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:06,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:06,485 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:34:08,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:10,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:12,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:34:13,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:34:14,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:16,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:18,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:20,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:22,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:24,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:26,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:28,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:29,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:34:29,100 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:34:30,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:32,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:34,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:36,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:36,563 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:34:38,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:40,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:42,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:44,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:44,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:34:44,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:34:46,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:48,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:50,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:52,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:54,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:56,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:34:58,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:00,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:35:00,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:35:02,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:04,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:06,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:35:07,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:09,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:11,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:13,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:15,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:35:15,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:35:17,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:19,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:21,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:23,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:25,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:27,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:29,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:30,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:35:30,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:35:31,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:33,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:35,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:36,716 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:35:37,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:39,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:41,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:43,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:45,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:35:46,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:35:47,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:49,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:51,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:53,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:55,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:57,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:35:59,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:01,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:01,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:36:01,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:36:03,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:05,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:06,792 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:36:07,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:09,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:11,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:13,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:15,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:16,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:36:16,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:36:17,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:19,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:21,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:23,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:25,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:28,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:30,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:36:31,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:36:32,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:34,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:36,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:36,871 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:36:38,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:40,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:42,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:44,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:46,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:36:47,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:36:48,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:50,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:52,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:54,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:56,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:36:58,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:00,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:02,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:02,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:37:02,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:37:04,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:06,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:06,948 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:37:08,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:10,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:12,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:14,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:16,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:37:17,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:37:18,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:20,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:22,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:24,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:26,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:28,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:30,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:32,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:33,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:37:33,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:37:34,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:36,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:37,024 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:37:38,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:40,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:42,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:44,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:46,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:48,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:37:48,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:37:49,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:51,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:53,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:55,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:57,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:37:59,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:01,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:03,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:03,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:38:03,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:38:05,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:07,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:38:07,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:09,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:11,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:13,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:15,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:17,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:18,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:38:18,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:38:19,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:21,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:23,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:25,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:27,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:29,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:31,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:33,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:34,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:38:34,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:38:35,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:37,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:38:37,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:39,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:41,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:43,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:46,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:48,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:49,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:38:49,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:38:50,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:52,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:54,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:56,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:38:58,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:00,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:02,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:04,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:04,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:39:04,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:39:06,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:07,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:39:08,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:10,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:12,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:14,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:16,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:18,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:39:19,884 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:39:20,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:22,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:24,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:26,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:28,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:30,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:32,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:34,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:39:35,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:39:36,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:37,334 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:39:38,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:40,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:42,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:44,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:46,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:48,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:50,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:50,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:39:50,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:39:52,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:54,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:56,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:39:58,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:00,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:02,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:04,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:05,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:40:05,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:40:06,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:07,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:40:08,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:10,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:12,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:14,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:16,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:18,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:21,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:40:21,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:40:21,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:23,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:25,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:27,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:29,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:31,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:33,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:35,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:36,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:40:36,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:40:37,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:37,484 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:40:39,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:41,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:43,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:45,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:47,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:49,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:51,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:51,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:40:51,573 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:40:53,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:55,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:57,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:40:59,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:01,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:03,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:05,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:06,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:41:06,826 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:41:07,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:07,558 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:41:09,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:11,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:13,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:15,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:17,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:19,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:21,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:41:22,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:41:23,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:25,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:28,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:30,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:32,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:34,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:36,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:37,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:41:37,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:41:37,686 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:41:38,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:40,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:42,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:44,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:46,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:48,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:50,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:52,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:52,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:41:52,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:41:54,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:56,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:41:58,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:00,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:02,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:04,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:06,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:07,707 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:42:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:42:07,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:42:08,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:10,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:12,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:14,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:16,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:18,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:20,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:22,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:23,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:42:23,208 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:42:24,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:26,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:28,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:30,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:32,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:34,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:36,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:37,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:42:38,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:42:38,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:42:40,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:42,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:44,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:46,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:48,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:50,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:52,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:42:53,934 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:42:54,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:56,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:42:58,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:01,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:03,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:05,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:07,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:07,856 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:43:09,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:43:09,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:43:09,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:11,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:13,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:15,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:17,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:19,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:21,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:23,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:24,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:43:24,594 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:43:25,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:27,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:29,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:31,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:33,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:35,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:37,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:37,934 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:43:39,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:39,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:43:39,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:43:41,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:43,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:45,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:47,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:49,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:51,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:53,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:55,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:43:55,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:43:55,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:57,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:43:59,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:01,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:04,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:06,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:08,010 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:44:08,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:10,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:10,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:44:10,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:44:12,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:14,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:16,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:18,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:20,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:22,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:24,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:25,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:44:25,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:44:26,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:28,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:30,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:32,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:34,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:36,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:38,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:44:38,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:40,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:44:40,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:44:42,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:44,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:46,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:48,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:50,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:52,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:54,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:56,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:44:56,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:44:56,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:44:58,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:00,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:02,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:04,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:06,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:08,158 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:45:08,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:10,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:11,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:45:11,565 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:45:12,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:14,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:16,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:18,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:20,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:22,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:25,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:26,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:45:26,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:45:27,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:29,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:31,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:33,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:35,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:37,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:38,232 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:45:39,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:41,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:45:42,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:45:43,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:45,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:47,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:49,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:51,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:53,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:55,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:57,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:45:57,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:45:57,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:45:59,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:01,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:03,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:05,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:07,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:08,306 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:46:09,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:11,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:46:12,796 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:46:13,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:15,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:17,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:19,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:21,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:23,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:25,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:27,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:46:28,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:46:29,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:31,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:34,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:36,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:38,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:46:38,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:40,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:42,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:46:43,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:46:44,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:46,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:48,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:50,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:52,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:54,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:56,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:58,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:46:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:46:58,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:47:00,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:02,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:04,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:06,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:08,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:08,460 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:47:10,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:12,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:13,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:47:13,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:47:14,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:16,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:18,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:20,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:22,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:24,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:26,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:28,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:29,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:47:29,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:47:30,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:32,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:34,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:36,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:38,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:38,534 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:47:40,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:42,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:44,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:47:44,516 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:47:46,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:48,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:50,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:52,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:54,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:56,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:58,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:47:59,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:47:59,748 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:48:01,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:03,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:05,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:07,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:08,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:48:09,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:11,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:13,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:14,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:48:14,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:48:15,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:17,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:19,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:21,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:23,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:25,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:27,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:29,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:48:30,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:48:31,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:33,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:35,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:37,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:38,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:48:39,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:41,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:43,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:45,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:45,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:48:45,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:48:47,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:49,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:51,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:53,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:55,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:57,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:48:59,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:01,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:49:01,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:49:01,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:03,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:05,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:07,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:08,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:49:09,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:11,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:13,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:15,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:16,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:49:16,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:49:17,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:19,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:21,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:23,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:25,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:27,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:29,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:31,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:31,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:49:31,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:49:33,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:35,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:37,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:38,831 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:49:39,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:41,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:44,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:46,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:49:47,059 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:49:48,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:50,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:52,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:54,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:56,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:49:58,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:00,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:02,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:50:02,403 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:50:02,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:04,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:06,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:08,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:08,905 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:50:10,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:12,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:14,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:16,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:17,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:50:17,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:50:18,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:20,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:22,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:24,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:26,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:28,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:30,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:32,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:33,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:50:33,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:50:34,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:36,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:38,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:38,976 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:50:40,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:42,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:44,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:46,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:48,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:50:48,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:50:48,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:50,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:52,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:54,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:56,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:50:58,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:00,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:02,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:51:03,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:51:04,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:06,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:08,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:09,052 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:51:10,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:12,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:14,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:16,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:18,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:18,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:51:18,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:51:20,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:22,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:24,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:26,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:28,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:30,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:32,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:34,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:51:34,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:51:34,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:36,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:38,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:39,127 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:51:40,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:42,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:44,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:46,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:48,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:49,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:51:49,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:51:50,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:52,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:54,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:56,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:51:58,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:00,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:02,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:04,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:52:04,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:52:05,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:07,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:09,202 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:52:09,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:11,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:13,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:15,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:17,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:19,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:52:20,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:52:21,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:23,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:25,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:27,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:29,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:31,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:33,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:52:35,367 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:52:35,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:37,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:39,279 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:52:39,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:41,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:43,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:45,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:47,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:49,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:50,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:52:50,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:52:51,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:53,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:55,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:57,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:52:59,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:01,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:03,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:05,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:05,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:53:05,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:53:07,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:09,356 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:53:09,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:11,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:13,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:15,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:17,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:19,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:21,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:53:21,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:53:21,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:23,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:25,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:27,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:29,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:31,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:33,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:35,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:36,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:53:36,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:53:37,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:39,432 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:53:40,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:42,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:44,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:46,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:48,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:50,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:51,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:53:51,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:53:52,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:54,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:56,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:53:58,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:00,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:02,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:04,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:06,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:07,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:54:07,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:54:08,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:09,510 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:54:10,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:12,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:14,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:16,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:18,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:20,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:22,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:54:22,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:54:22,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:24,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:26,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:28,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:30,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:32,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:34,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:36,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:54:37,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:54:38,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:39,585 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:54:40,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:42,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:44,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:46,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:48,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:50,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:52,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:52,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:54:52,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:54:54,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:57,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:54:59,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:01,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:03,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:05,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:07,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:08,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:55:08,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:55:09,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:09,662 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:55:11,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:13,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:15,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:17,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:19,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:21,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:23,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:23,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:55:23,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:55:25,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:27,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:29,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:31,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:33,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:35,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:37,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:39,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:55:39,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:55:39,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:39,746 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:55:41,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:43,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:45,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:47,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:49,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:51,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:53,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:55:54,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:55:55,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:57,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:55:59,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:01,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:03,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:05,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:07,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:09,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:09,822 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:56:10,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:56:10,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:56:11,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:13,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:15,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:17,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:19,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:21,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:23,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:25,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:56:25,307 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:56:25,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:27,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:29,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:31,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:33,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:35,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:56:39,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:56:40,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:56:40,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:56:56,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:56:56,261 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:57:09,975 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:57:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:57:11,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:57:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:57:26,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:57:34,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 10:57:34,314 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 10:57:34,367 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 10:57:34,444 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 10:57:34,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 10:57:35,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:57:36,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:57:40,061 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:57:40,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:57:42,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:57:42,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:57:57,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:57:57,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:58:00,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 10:58:10,142 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:58:12,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:58:12,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:58:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:58:27,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:58:40,221 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:58:43,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:58:43,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:58:58,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:58:58,375 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:59:10,296 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:59:13,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:59:13,604 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:59:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:59:28,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:59:40,375 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 10:59:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:59:44,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 10:59:59,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 10:59:59,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:00:10,453 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:00:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:00:14,595 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:00:29,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:00:29,835 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:00:40,529 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:00:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:00:45,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:01:00,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:01:00,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:01:10,603 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:01:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:01:15,632 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:01:30,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:01:30,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:01:40,679 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:01:46,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:01:46,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:01:54,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:01:56,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:56,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:58,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:01:59,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:01:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:00,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:01,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:02:01,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:02:02,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:02,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:04,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:06,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:08,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:08,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:10,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:10,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:02:10,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:12,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:13,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:13,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:14,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:02:16,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:02:16,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:18,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:20,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:22,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:22,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:24,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:25,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:26,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:28,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:28,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:30,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:31,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:02:31,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:02:32,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:33,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:34,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:34,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:37,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:38,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:38,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:39,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:40,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:40,831 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:02:41,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:42,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:43,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:43,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:44,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:45,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:02:47,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:02:56,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:56,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:57,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:02:58,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:02:58,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:01,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:02,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:02,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:03:02,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:03:03,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:03,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:05,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:07,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:09,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:10,901 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:03:11,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:11,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:13,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:03:17,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:03:17,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:21,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:23,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:24,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:26,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:27,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:29,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:33,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:03:33,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:03:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:33,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:35,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:39,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:40,977 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:03:41,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:41,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:41,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:42,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:43,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:45,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:45,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:48,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:03:48,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:03:48,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:48,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:49,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:50,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:51,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:53,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:53,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:53,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:55,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:57,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:03:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:58,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:03:59,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:00,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:01,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:03,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:04:03,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:04:03,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:05,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:07,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:07,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:09,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:09,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:11,048 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:04:11,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:11,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:13,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:13,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:15,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:17,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:19,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:04:19,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:04:19,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:21,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:23,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:24,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:26,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:28,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:30,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:30,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:32,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:32,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:32,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:34,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:04:34,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:04:34,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:36,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:36,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:37,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:38,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:39,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:39,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:40,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:41,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:04:44,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:44,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:49,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:04:49,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:04:50,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:04:53,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:04:56,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:00,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:01,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:03,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:04,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:04,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:05:04,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:05:05,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:05,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:08,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:09,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:11,178 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:05:12,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:13,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:14,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:18,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:05:20,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:05:22,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:24,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:24,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:26,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:26,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:28,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:30,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:32,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:32,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:34,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:35,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:05:35,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:05:36,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:36,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:40,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:41,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:05:42,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:44,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:45,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:46,486 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 11:05:46,695 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 11:05:46,755 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 11:05:46,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 11:05:47,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:47,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:48,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:05:50,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:05:50,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:50,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:52,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:54,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:54,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:56,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:05:57,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:57,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:05:59,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:00,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:01,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:02,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:04,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:04,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:06:06,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:06:06,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:08,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:08,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:08,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:10,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:11,338 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:06:12,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:13,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:13,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:14,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:16,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:18,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:20,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:21,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:06:21,373 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:06:22,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:24,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:25,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:26,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:26,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:28,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:30,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:31,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:31,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:32,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:33,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:34,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:36,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:06:36,658 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:06:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:36,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:37,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:39,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:41,402 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:06:41,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:48,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:51,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:52,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:06:52,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:06:55,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:06:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:06:59,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:03,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:07,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:07:07,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:07:07,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:09,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:09,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:10,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:11,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:07:11,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:12,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:12,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:15,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:17,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:18,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:21,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:22,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:07:22,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:07:25,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:27,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:31,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:32,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:32,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:33,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:34,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:36,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:37,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:07:37,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:07:37,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:39,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:41,543 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:07:41,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:43,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:44,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:47,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:49,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:51,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:53,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:07:53,099 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:07:53,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:55,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:56,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:56,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:57,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:07:58,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:58,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:07:59,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:01,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:02,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:02,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:03,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:05,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:06,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:06,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:07,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:08,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:08:08,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:08:09,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:11,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:11,616 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:08:11,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:13,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:15,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:16,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:16,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:17,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:19,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:21,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:23,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:08:23,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:08:23,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:24,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:25,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:26,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:27,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:28,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:29,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:31,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:32,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:32,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:33,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:34,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:34,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:35,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:37,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:38,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:08:38,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:08:39,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:41,684 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:08:44,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:44,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:45,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:46,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:51,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:53,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:08:54,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:08:55,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:08:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:08:59,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:01,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:03,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:04,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:04,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:05,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:09:09,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:09:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:09,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:11,759 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:09:13,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:14,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:14,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:16,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:19,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:20,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:22,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:09:24,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:09:25,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:26,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:28,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:30,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:31,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:31,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:34,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:36,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:38,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:38,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:38,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:38,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:38,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:39,677 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 11:09:39,883 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 11:09:39,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:09:39,950 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 11:09:39,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:09:39,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 11:09:40,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:41,956 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:09:41,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:43,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:43,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:44,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:44,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:45,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:46,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:48,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:50,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:50,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:51,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:52,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:53,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:53,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:54,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:55,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:09:55,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:09:56,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:09:58,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:09:58,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:00,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:02,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:03,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:04,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:05,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:05,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:06,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:08,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:08,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:10,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:10:10,600 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:10:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:10,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:12,027 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:10:12,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:12,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:15,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:17,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:19,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:20,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:21,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:23,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:25,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:10:25,934 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:10:27,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:27,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:27,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:29,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:31,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:40,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:41,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:41,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:10:41,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:10:42,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:10:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:42,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:45,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:47,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:47,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:48,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:49,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:53,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:10:55,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:55,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:10:56,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:10:56,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:10:57,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:01,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:05,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:07,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:07,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:08,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:11,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:11,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:11:11,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:11:12,185 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:11:13,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:17,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:19,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:20,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:20,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:26,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:26,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:11:26,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:11:27,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:27,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:28,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:30,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:31,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:32,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:33,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:33,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:34,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:36,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:38,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:40,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:41,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:41,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,257 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:11:42,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:11:42,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:11:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:42,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:44,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:44,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:46,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:48,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:50,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:50,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:52,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:52,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:54,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:54,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:56,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:11:57,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:11:58,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:11:58,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:11:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:00,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:01,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:02,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:02,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:03,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:04,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:05,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:06,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:07,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:08,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:10,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:10,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:10,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:11,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:11,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:11,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:11,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:12,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:12,330 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:12:12,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:12:12,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:12:14,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:16,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:16,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:18,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:18,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:20,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:21,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:23,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:24,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:26,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:12:28,126 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:12:36,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:38,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:42,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:42,402 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:12:43,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:12:43,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:12:45,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:45,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:46,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:50,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:52,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:52,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:52,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:53,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:12:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:56,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:58,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:12:58,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:12:59,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:12:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:00,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:04,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:06,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:09,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:10,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:10,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:12,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:12,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:13:14,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:13:14,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:13:16,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:16,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:17,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:19,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:20,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:23,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:25,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:27,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:13:29,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:13:30,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:30,828 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 11:13:31,030 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 11:13:31,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:31,111 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 11:13:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:31,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:32,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 11:13:32,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:33,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:35,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:37,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:37,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:38,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:39,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:39,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:42,570 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:13:43,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:44,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:13:44,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:13:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:45,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:45,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:47,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:48,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:48,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:49,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:51,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:51,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:53,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:54,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:54,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:56,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:57,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:13:59,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:13:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:13:59,977 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:14:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:00,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:01,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:01,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:03,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:04,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:05,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:06,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:06,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:07,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:09,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:11,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:12,642 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:14:13,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:15,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:14:15,251 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:14:17,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:19,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:20,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:21,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:23,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:28,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:28,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:30,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:14:30,491 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:14:31,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:35,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:39,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:42,717 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:14:43,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:44,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:45,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:14:45,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:14:47,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:47,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:48,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:51,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:53,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:57,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:14:59,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:14:59,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:00,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:15:00,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:15:03,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:05,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:08,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:09,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:09,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:11,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:12,787 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:15:14,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:14,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:15,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:15:16,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:15:17,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:17,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:17,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:19,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:21,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:21,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:22,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:25,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:25,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:27,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:28,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:29,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:30,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:31,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:15:31,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:15:33,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:35,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:35,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:37,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:37,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:41,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:42,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:15:43,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:44,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:45,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:45,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:46,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:15:46,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:15:48,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:48,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:50,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:51,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:52,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:52,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:54,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:55,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:55,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:56,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:57,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:58,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:15:59,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:15:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:00,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:01,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:01,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:02,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:16:02,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:16:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:02,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:04,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:06,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:08,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:08,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:10,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:11,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:11,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:12,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:12,938 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:16:13,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:13,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:14,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:16,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:16:17,528 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:16:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:23,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:26,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:29,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:29,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:32,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:32,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:16:32,845 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:16:36,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:40,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:41,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:41,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:43,013 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:16:44,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:16:48,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:16:48,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:52,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:54,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:16:54,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:57,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:16:58,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:00,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:00,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:00,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:02,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:17:03,335 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:17:03,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:04,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:06,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:10,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:12,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:13,082 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:17:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:14,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:16,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:16,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:18,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:18,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:17:18,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:17:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:19,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:20,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:21,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:21,791 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 11:17:21,970 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 11:17:22,041 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 11:17:22,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 11:17:22,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:23,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:24,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:24,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:26,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:28,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:29,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:29,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:30,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:31,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:31,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:32,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:33,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:17:33,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:17:34,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:34,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:36,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:37,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:37,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:38,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:38,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:40,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:41,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:42,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:43,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:43,191 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:17:44,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:44,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:46,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:46,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:47,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:48,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:49,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:17:49,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:17:50,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:50,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:51,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:51,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:52,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:53,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:56,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:57,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:58,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:17:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:17:59,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:00,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:01,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:01,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:02,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:03,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:03,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:04,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:04,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:18:04,346 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:18:06,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:06,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:08,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:09,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:10,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:11,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:12,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:13,256 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:18:14,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:19,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:18:19,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:18:22,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:23,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:23,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:29,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:30,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:33,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:18:35,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:18:37,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:41,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:43,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:18:45,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:47,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:18:50,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:18:51,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:53,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:54,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:54,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:56,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:18:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:57,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:18:59,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:03,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:03,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:03,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:05,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:05,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:19:05,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:19:09,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:11,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:12,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:13,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:13,394 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:19:15,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:15,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:15,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:15,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:15,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:16,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:19:21,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:19:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:21,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:22,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:22,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:22,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:23,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:24,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:24,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:25,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:27,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:29,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:29,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:29,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:31,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:32,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:32,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:33,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:35,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:35,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:19:36,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:19:37,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:37,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:37,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:38,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:39,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:41,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:41,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:42,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:43,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:43,462 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:19:44,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:44,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:45,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:46,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:47,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:48,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:48,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:49,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:51,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:51,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:19:51,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:19:52,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:52,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:53,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:53,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:54,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:55,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:19:57,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:57,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:58,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:19:59,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:00,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:00,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:01,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:01,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:03,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:05,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:05,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:06,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:07,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:20:07,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:20:07,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:13,540 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:20:17,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:21,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:20:22,375 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:20:24,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:24,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:26,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:26,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:30,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:34,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:37,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:20:37,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:20:38,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:42,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:43,613 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:20:44,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:48,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:52,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:53,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:20:53,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:20:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:53,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:54,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:56,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:20:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:56,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:20:59,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:00,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:02,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:05,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:05,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:06,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:08,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:21:08,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:21:10,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:10,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:10,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:10,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 11:21:11,763 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 11:21:11,975 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 11:21:12,044 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 11:21:12,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 11:21:12,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:13,694 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:21:14,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:16,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:18,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:20,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:22,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:23,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:21:23,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:21:24,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:26,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:28,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:30,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:32,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:34,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:36,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:38,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:39,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:21:39,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:21:40,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:42,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:43,767 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:21:44,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:46,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:48,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:50,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:52,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:54,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:21:54,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:21:56,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:21:58,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:00,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:03,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:05,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:07,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:09,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:09,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:22:09,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:22:11,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:13,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:13,840 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:22:15,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:17,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:19,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:21,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:23,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:22:24,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:22:25,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:27,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:29,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:31,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:33,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:35,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:37,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:39,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:40,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:22:40,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:22:41,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:43,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:43,904 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:22:45,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:47,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:49,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:51,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:53,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:22:55,595 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:22:56,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:22:58,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:00,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:02,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:04,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:06,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:08,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:10,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:23:10,837 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:23:12,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:13,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:23:14,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:16,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:18,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:20,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:22,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:24,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:23:26,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:23:26,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:28,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:30,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:32,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:34,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:36,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:38,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:40,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:23:41,530 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:23:42,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:44,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:23:44,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:46,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:48,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:50,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:52,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:54,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:56,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:23:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:23:56,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:23:58,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:00,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:02,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:04,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:06,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:08,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:10,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:24:12,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:24:12,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:14,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:24:14,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:16,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:18,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:20,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:22,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:24,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:26,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:27,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:24:27,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:24:28,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:31,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:33,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:35,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:37,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:39,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:41,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:24:42,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:24:43,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:44,196 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:24:45,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:47,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:49,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:51,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:53,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:55,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:57,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:24:57,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:24:57,996 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:24:59,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:01,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:03,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:05,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:07,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:09,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:11,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:13,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:25:13,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:25:13,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:14,270 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:25:15,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:17,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:19,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:21,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:23,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:26,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:28,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:28,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:25:28,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:25:30,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:32,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:34,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:36,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:38,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:40,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:42,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:43,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:25:43,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:25:44,346 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:25:44,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:46,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:48,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:50,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:52,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:54,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:56,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:58,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:25:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:25:59,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:26:00,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:02,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:04,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:06,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:08,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:10,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:12,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:14,422 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:26:14,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:26:14,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:26:16,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:18,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:20,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:22,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:24,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:26,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:28,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:29,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:26:29,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:26:30,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:32,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:34,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:37,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:39,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:41,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:43,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:44,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:26:45,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:26:45,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:26:45,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:47,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:49,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:51,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:53,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:55,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:57,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:26:59,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:27:00,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:27:01,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:03,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:05,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:07,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:09,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:11,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:13,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:14,572 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:27:15,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:15,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:27:15,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:27:17,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:19,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:21,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:23,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:25,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:27,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:29,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:30,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:27:30,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:27:31,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:33,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:35,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:37,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:39,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:41,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:43,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:44,646 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:27:45,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:27:46,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:27:47,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:49,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:51,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:53,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:56,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:27:58,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:00,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:01,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:28:01,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:28:02,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:04,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:06,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:08,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:10,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:12,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:14,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:14,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:28:16,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:16,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:28:16,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:28:18,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:20,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:22,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:24,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:26,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:28,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:30,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:32,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:28:32,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:28:32,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:34,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:36,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:38,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:40,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:42,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:44,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:44,794 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:28:46,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:47,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:28:47,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:28:48,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:50,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:52,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:54,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:56,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:28:58,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:00,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:02,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:02,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:29:02,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:29:04,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:06,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:08,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:10,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:12,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:14,870 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:29:15,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:17,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:17,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:29:17,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:29:19,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:21,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:23,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:25,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:27,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:29,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:31,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:29:33,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:29:33,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:35,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:37,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:39,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:41,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:43,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:44,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:29:45,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:47,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:48,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:29:48,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:29:49,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:51,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:53,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:55,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:57,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:29:59,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:01,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:03,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:03,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:30:03,806 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:30:05,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:07,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:09,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:11,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:13,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:15,023 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:30:15,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:17,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:30:19,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:30:19,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:21,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:23,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:25,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:27,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:29,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:31,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:33,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:30:34,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:30:35,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:37,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:39,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:41,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:43,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:45,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:30:45,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:47,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:49,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:50,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:30:50,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:30:51,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:53,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:55,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:57,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:30:59,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:01,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:03,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:05,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:31:05,409 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:31:05,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:07,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:09,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:11,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:13,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:15,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:31:16,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:18,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:20,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:31:20,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:31:22,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:24,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:26,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:28,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:30,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:32,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:34,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:31:35,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:31:36,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:38,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:40,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:42,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:44,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:45,246 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:31:46,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:48,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:50,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:51,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:31:51,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:31:52,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:54,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:56,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:31:58,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:00,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:02,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:04,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:06,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:32:06,528 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:32:06,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:08,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:10,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:12,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:14,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:15,321 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:32:16,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:18,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:20,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:21,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:32:21,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:32:22,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:24,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:26,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:28,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:30,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:32,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:34,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:37,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:32:37,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:32:37,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:39,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:41,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:43,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:45,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:32:45,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:47,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:49,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:51,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:52,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:32:52,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:32:53,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:55,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:57,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:32:59,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:01,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:03,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:05,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:07,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:33:07,530 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:33:07,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:09,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:11,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:13,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:15,467 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:33:15,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:17,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:19,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:21,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:22,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:33:22,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:33:23,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:25,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:27,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:29,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:31,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:33,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:35,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:37,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:38,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:33:38,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:33:39,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:41,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:43,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:45,539 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:33:45,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:47,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:49,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:51,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:53,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:53,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:33:53,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:33:55,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:57,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:33:59,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:01,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:03,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:05,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:08,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:09,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:34:09,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:34:10,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:12,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:14,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:15,613 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:34:16,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:18,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:20,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:22,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:24,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:34:24,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:34:24,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:26,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:28,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:30,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:32,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:34,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:36,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:38,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:39,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:34:39,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:34:40,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:42,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:44,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:45,687 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:34:46,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:48,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:50,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:52,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:54,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:54,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:34:54,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:34:56,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:34:58,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:00,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:02,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:04,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:06,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:08,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:35:10,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:35:10,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:12,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:14,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:15,762 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:35:16,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:18,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:20,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:22,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:24,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:25,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:35:25,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:35:26,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:28,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:31,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:33,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:35,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:37,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:39,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:40,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:35:40,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:35:41,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:43,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:45,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:45,839 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:35:47,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:49,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:51,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:53,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:55,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:56,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:35:56,107 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:35:57,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:35:59,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:01,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:03,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:05,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:07,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:09,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:11,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:36:11,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:36:11,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:13,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:15,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:15,914 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:36:17,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:19,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:21,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:23,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:25,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:26,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:36:26,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:36:27,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:29,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:31,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:33,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:35,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:37,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:39,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:41,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:42,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:36:42,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:36:43,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:45,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:45,992 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:36:47,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:49,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:51,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:53,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:55,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:36:57,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:36:57,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:36:59,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:01,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:03,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:05,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:07,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:09,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:11,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:37:12,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:37:13,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:15,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:16,068 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:37:18,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:20,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:22,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:24,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:26,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:27,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:37:27,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:37:28,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:30,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:32,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:34,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:36,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:38,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:40,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:42,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:43,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:37:43,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:37:44,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:46,146 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:37:46,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:48,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:50,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:52,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:54,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:56,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:37:58,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:37:58,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:37:58,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:00,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:02,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:04,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:06,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:08,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:10,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:12,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:13,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:38:13,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:38:14,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:16,219 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:38:16,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:18,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:20,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:22,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:24,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:26,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:28,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:38:28,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:38:30,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:33,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:35,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:37,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:39,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:41,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:43,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:44,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:38:44,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:38:45,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:46,295 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:38:47,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:49,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:51,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:53,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:55,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:57,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:38:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:38:59,401 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:38:59,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:01,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:03,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:05,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:07,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:09,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:11,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:13,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:14,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:39:14,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:39:15,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:16,371 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:39:17,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:19,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:21,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:23,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:25,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:27,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:29,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:29,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:39:29,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:39:31,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:33,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:35,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:37,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:39,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:41,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:43,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:39:45,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:39:45,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:46,447 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:39:48,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:50,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:52,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:54,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:56,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:39:58,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:00,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:40:00,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:40:00,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:02,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:04,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:06,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:08,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:10,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:12,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:14,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:40:15,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:40:16,523 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:40:16,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:18,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:20,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:22,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:24,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:26,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:28,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:30,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:31,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:40:31,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:40:32,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:34,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:36,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:38,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:40,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:42,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:44,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:40:46,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:40:46,690 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:40:46,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:48,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:50,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:52,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:54,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:56,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:40:58,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:00,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:01,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:41:01,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:41:02,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:04,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:06,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:08,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:10,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:12,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:14,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:16,674 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:41:16,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:41:17,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:41:18,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:20,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:22,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:24,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:26,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:28,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:30,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:32,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:41:32,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:41:32,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:34,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:36,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:38,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:40,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:42,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:44,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:46,750 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:41:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:41:47,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:41:47,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:49,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:51,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:53,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:55,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:57,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:41:59,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:01,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:02,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:42:02,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:42:03,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:05,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:07,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:09,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:11,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:13,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:15,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:16,825 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:42:17,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:18,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:42:18,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:42:19,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:21,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:23,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:25,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:27,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:29,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:31,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:33,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:42:33,566 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:42:33,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:35,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:37,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:39,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:41,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:43,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:45,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:46,901 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:42:47,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:48,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:42:48,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:42:49,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:51,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:53,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:55,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:57,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:42:59,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:01,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:03,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:04,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:43:04,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:43:05,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:08,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:10,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:12,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:14,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:16,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:16,979 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:43:18,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:19,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:43:19,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:43:20,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:22,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:24,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:26,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:28,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:30,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:32,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:43:34,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:43:34,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:36,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:38,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:40,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:42,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:44,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:46,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:47,052 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:43:48,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:50,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:43:50,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:43:50,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:52,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:54,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:56,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:43:58,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:00,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:02,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:04,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:06,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:44:06,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:44:06,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:08,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:10,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:12,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:14,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:16,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:17,126 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:44:18,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:21,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:44:21,860 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:44:21,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:23,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:25,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:27,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:29,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:31,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:33,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:35,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:37,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:44:37,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:44:37,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:39,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:41,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:43,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:45,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:47,199 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:44:47,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:49,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:51,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:44:52,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:44:53,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:55,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:57,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:44:59,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:01,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:03,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:05,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:07,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:45:07,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:45:07,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:09,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:11,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:13,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:15,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:17,273 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:45:17,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:19,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:21,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:23,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:45:23,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:45:23,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:25,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:27,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:29,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:31,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:33,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:35,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:37,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:38,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:45:38,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:45:39,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:41,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:43,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:45,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:47,347 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:45:47,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:49,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:51,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:53,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:45:53,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:45:53,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:55,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:57,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:45:59,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:01,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:03,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:05,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:07,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:08,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:46:08,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:46:09,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:12,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:14,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:16,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:17,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:46:18,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:20,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:22,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:24,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:46:24,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:46:24,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:26,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:28,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:30,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:32,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:34,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:36,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:38,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:46:39,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:46:40,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:42,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:44,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:46,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:47,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:46:48,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:50,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:52,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:54,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:46:54,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:46:54,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:56,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:46:58,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:00,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:02,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:04,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:06,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:08,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:47:10,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:47:10,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:12,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:14,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:16,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:17,552 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:47:18,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:20,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:22,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:24,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:25,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:47:25,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:47:26,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:28,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:30,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:32,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:34,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:36,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:38,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:47:40,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:47:40,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:42,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:44,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:46,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:47,630 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:47:48,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:50,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:52,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:54,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:55,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:47:55,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:47:56,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:47:58,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:00,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:02,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:05,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:07,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:09,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:11,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:48:11,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:48:11,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:13,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:15,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:17,704 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:48:17,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:19,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:21,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:23,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:25,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:26,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:48:26,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:48:27,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:29,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:31,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:33,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:35,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:37,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:39,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:41,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:48:41,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:48:41,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:43,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:45,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:47,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:48:47,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:49,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:51,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:53,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:55,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:48:56,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:48:57,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:48:59,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:01,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:04,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:06,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:08,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:10,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:12,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:49:12,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:49:14,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:16,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:17,858 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:49:18,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:20,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:22,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:24,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:26,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:27,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:49:27,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:49:28,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:30,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:32,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:34,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:36,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:38,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:40,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:42,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:42,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:49:42,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:49:44,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:47,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:47,935 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:49:49,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:51,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:53,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:55,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:57,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:49:57,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:49:57,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:49:59,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:01,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:03,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:05,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:07,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:09,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:11,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:13,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:13,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:50:13,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:50:15,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:17,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:18,010 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:50:19,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:21,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:23,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:25,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:27,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:28,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:50:28,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:50:29,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:31,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:33,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:35,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:37,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:39,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:41,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:43,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:50:43,851 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:50:46,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:48,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:48,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:50:50,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:52,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:54,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:56,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:58,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:50:59,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:50:59,127 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:51:00,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:02,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:04,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:06,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:08,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:10,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:12,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:14,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:14,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:51:14,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:51:16,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:18,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:18,143 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:51:20,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:22,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:24,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:26,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:28,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:29,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:51:29,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:51:30,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:32,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:34,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:36,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:38,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:40,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:42,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:44,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:45,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:51:45,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:51:46,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:48,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:48,214 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:51:50,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:52,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:54,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:56,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:51:58,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:00,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:52:00,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:52:02,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:04,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:06,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:08,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:10,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:12,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:14,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:15,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:52:15,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:52:16,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:18,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:18,283 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:52:20,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:22,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:24,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:26,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:28,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:30,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:30,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:52:30,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:52:32,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:34,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:36,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:38,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:40,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:42,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:45,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:46,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:52:46,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:52:47,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:48,350 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:52:49,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:51,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:53,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:55,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:57,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:52:59,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:01,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:01,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:53:01,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:53:03,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:05,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:07,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:09,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:11,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:13,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:15,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:16,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:53:16,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:53:17,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:18,418 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:53:19,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:21,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:23,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:25,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:27,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:29,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:31,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:32,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:53:32,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:53:33,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:35,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:37,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:39,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:41,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:43,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:45,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:47,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:53:47,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:53:48,481 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:53:49,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:51,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:53,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:55,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:57,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:53:59,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:01,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:54:02,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:54:03,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:05,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:07,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:09,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:11,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:13,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:15,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:17,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:54:17,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:54:18,546 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:54:19,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:21,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:23,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:25,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:28,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:30,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:32,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:54:33,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:54:34,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:36,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:38,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:40,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:42,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:44,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:46,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:48,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:48,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:54:48,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:54:48,665 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:54:50,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:52,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:54,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:56,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:54:58,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:00,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:02,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:03,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:55:03,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:55:04,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:06,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:08,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:10,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:12,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:14,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:16,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:18,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:18,695 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:55:19,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:55:19,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:55:20,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:22,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:24,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:26,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:28,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:30,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:32,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:34,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:34,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:55:34,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:55:36,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:38,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:40,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:42,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:44,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:46,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:48,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:48,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:55:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:55:49,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:55:50,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:52,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:54,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:56,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:55:59,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:01,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:03,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:05,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:56:05,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:56:05,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:07,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:09,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:11,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:13,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:15,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:17,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:18,849 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:56:19,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:20,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:56:20,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:56:21,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:23,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:25,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:27,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:29,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:31,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:33,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:35,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:56:35,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:56:37,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:39,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:41,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:43,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:45,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:47,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:48,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:56:49,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:51,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:51,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:56:51,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:56:53,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:55,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:57,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:56:59,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:02,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:04,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:06,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:06,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:57:06,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:57:08,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:10,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:12,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:14,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:16,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:18,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:19,002 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:57:20,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:21,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:57:21,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:57:22,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:24,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:26,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:28,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:30,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:32,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:34,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:36,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:57:37,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:57:38,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:40,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:42,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:44,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:46,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:48,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:49,078 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:57:50,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:52,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:52,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:57:52,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:57:54,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:56,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:57:58,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:00,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:02,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:04,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:06,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:58:07,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:58:08,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:10,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:12,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:14,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:16,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:18,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:19,156 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:58:20,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:22,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:23,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:58:23,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:58:24,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:26,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:28,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:30,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:32,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:34,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:36,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:38,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:38,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:58:38,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:58:40,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:42,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:45,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:47,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:49,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:58:49,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:51,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:53,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:53,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:58:53,602 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:58:55,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:57,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:58:59,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:01,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:03,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:05,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:07,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:09,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:59:09,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:59:11,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:13,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:15,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:17,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:19,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:19,310 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:59:21,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:23,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:24,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:59:24,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:59:25,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:27,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:29,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:31,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:33,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:35,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:37,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:39,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:40,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:59:40,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:59:41,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:43,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:45,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:47,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:49,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:49,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 11:59:51,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:53,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:55,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 11:59:55,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 11:59:55,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 11:59:57,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:00,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:02,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:04,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:06,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:08,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:10,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:10,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:00:10,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:00:12,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:14,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:16,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:18,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:19,466 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:00:20,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:22,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:24,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:25,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:00:25,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:00:26,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:28,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:30,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:32,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:34,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:36,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:38,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:40,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:41,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:00:41,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:00:42,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:44,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:46,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:48,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:49,543 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:00:50,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:52,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:54,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:56,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:00:57,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:00:57,091 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:00:58,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:00,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:02,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:04,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:07,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:09,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:11,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:01:12,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:01:13,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:15,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:17,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:19,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:19,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:01:21,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:23,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:25,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:27,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:28,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:01:28,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:01:29,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:31,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:33,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:35,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:37,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:39,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:41,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:43,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:43,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:01:43,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:01:45,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:47,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:49,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:49,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:01:51,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:53,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:55,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:57,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:01:58,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:01:58,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:01:59,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:01,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:03,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:05,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:07,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:09,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:11,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:13,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:14,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:02:14,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:02:15,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:17,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:19,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:19,779 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:02:21,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:23,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:25,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:27,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:29,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:29,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:02:29,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:02:31,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:33,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:36,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:38,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:40,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:42,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:44,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:44,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:02:44,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:02:46,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:48,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:49,856 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:02:50,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:52,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:54,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:56,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:02:58,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:03:00,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:03:00,003 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:03:00,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:03:15,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:03:15,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:03:19,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:03:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:03:30,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:03:45,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:03:45,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:03:50,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:03:58,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:03:58,953 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 12:03:59,004 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 12:03:59,079 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 12:03:59,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 12:04:00,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:04:01,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:04:01,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:04:01,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:04:06,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:04:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:04:16,273 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:04:20,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:04:22,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:04:31,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:04:31,530 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:04:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:04:46,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:04:50,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:05:02,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:05:02,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:05:17,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:05:17,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:05:20,249 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:05:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:05:32,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:05:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:05:47,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:05:50,326 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:06:03,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:06:03,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:06:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:06:18,293 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:06:20,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:06:33,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:06:33,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:06:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:06:48,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:06:50,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:07:04,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:07:04,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:07:19,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:07:19,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:07:20,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:07:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:07:34,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:07:49,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:07:49,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:07:50,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:08:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:08:05,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:08:16,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:20,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:20,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:08:20,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:08:20,759 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:08:22,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:22,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:24,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:25,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:25,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:27,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:28,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:30,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:31,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:33,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:33,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:35,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:35,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:35,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:08:35,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:08:37,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:37,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:39,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:41,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:42,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:42,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:43,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:45,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:46,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:47,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:47,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:48,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:49,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:50,784 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:08:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:50,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:08:51,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:08:51,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:53,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:55,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:56,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:57,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:08:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:08:59,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:00,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:00,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:01,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:02,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:02,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:03,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:05,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:09:06,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:09:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:07,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:09,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:17,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:18,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:18,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:20,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:09:21,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:09:21,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:09:23,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:27,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:31,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:32,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:34,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:35,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:36,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:09:36,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:09:37,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:41,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:43,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:47,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:49,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:50,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:09:52,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:09:52,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:09:52,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:53,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:55,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:57,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:09:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:09:57,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:01,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:03,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:03,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:05,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:07,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:10:07,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:10:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:09,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:09,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:11,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:12,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:12,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:13,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:15,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:17,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:19,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:19,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:19,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,015 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:10:21,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:21,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:10:22,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:10:22,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:25,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:27,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:29,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:30,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:30,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:31,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:33,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:33,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:33,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:35,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:35,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:37,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:38,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:10:38,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:10:38,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:40,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:40,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:42,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:44,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:45,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:46,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:48,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:49,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:50,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:50,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,087 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:10:51,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:52,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:53,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:10:53,357 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:10:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:54,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:55,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:56,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:57,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:57,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:10:58,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:58,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:10:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:00,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:02,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:03,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:08,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:11:08,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:11:12,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:16,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:20,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:20,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:21,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:11:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:11:23,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:11:24,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:28,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:31,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:31,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:32,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:34,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:36,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:36,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:38,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:39,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:11:39,122 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:11:42,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:44,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:48,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:50,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:51,238 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:11:54,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:11:54,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:11:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:54,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:56,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:58,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:11:59,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:11:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:02,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:02,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:04,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:04,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:05,624 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 12:12:05,828 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 12:12:05,895 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 12:12:06,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 12:12:06,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:09,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:12:09,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:12:10,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:10,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:12,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:14,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:14,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:16,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:18,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:20,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:20,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:21,333 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:12:22,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:22,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:24,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:24,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:12:24,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:12:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:25,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:26,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:28,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:28,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:30,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:31,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:31,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:32,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:33,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:34,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:36,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:37,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:37,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:38,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:40,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:12:40,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:12:40,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:42,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:42,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:44,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:46,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:47,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:47,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:48,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:50,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:51,409 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:12:52,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:52,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:53,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:53,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:53,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:53,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:54,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:55,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:12:55,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:12:55,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:12:57,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:12:59,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:01,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:07,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:10,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:13:10,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:13:13,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:13,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:14,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:14,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:17,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:20,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:20,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:21,486 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:13:21,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:25,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:25,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:13:25,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:13:29,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:31,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:34,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:34,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:35,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:37,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:39,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:13:41,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:13:41,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:43,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:47,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:47,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:49,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:51,563 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:13:51,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:55,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:56,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:13:56,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:13:57,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:13:58,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:58,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:13:59,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:00,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:00,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:01,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:05,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:07,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:09,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:11,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:11,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:11,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:14:11,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:14:12,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:12,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:13,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:15,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:16,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:16,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:17,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:19,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:21,638 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:14:21,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:21,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:21,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:23,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:24,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:25,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:26,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:26,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:14:27,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:14:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:27,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:28,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:28,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:29,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:31,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:32,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:32,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:33,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:34,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:35,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:36,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:36,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:37,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:39,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:39,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:40,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:41,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:41,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:42,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:14:42,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:14:42,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:43,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:44,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:46,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:46,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:47,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:47,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:48,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:49,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:49,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:50,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:51,715 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:14:52,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:14:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:57,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:14:57,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:14:59,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:14:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:00,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:04,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:08,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:08,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:08,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:12,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:13,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:15:13,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:15:13,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:13,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:16,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:20,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:21,791 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:15:22,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:24,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:26,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:28,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:15:28,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:15:30,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:32,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:33,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:36,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:38,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:40,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:40,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:15:43,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:15:44,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:46,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:48,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:50,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:51,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,867 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:15:51,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:51,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:52,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:52,124 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 12:15:52,319 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 12:15:52,394 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 12:15:52,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 12:15:53,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:53,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:53,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:54,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:56,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:58,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:15:58,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:15:59,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:15:59,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:16:00,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:02,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:02,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:02,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:04,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:06,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:06,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:07,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:08,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:10,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:12,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:12,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:16:14,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:16:14,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:15,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:15,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:16,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:18,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:18,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:20,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:21,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:16:22,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:23,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:23,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:24,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:26,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:26,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:27,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:28,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:29,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:29,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:16:29,851 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:16:30,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:30,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:32,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:32,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:33,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:33,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:34,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:36,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:37,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:38,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:40,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:40,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:42,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:16:45,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:16:47,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:47,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:48,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:49,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:52,042 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:16:54,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:54,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:16:56,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:16:58,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:17:00,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:17:02,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:02,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:04,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:04,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:06,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:10,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:10,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:14,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:16,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:17:16,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:17:16,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:18,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:20,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:22,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:17:22,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:24,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:24,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:26,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:28,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:31,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:17:31,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:17:31,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:31,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:32,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:33,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:35,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:37,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:39,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:41,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:43,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:43,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:45,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:46,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:17:46,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:17:47,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:49,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:49,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:52,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:17:53,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:54,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:54,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:55,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:57,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:57,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:17:58,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:58,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:17:59,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:01,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:18:01,813 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:18:03,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:03,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:03,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:04,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:05,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:07,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:09,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:11,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:11,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:13,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:13,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:13,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:15,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:16,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:16,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:18:17,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:18:17,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:17,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:17,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:19,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:21,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:22,272 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:18:22,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:23,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:24,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:25,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:26,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:26,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:27,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:28,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:28,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:29,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:30,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:30,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:31,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:32,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:18:32,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:18:33,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:34,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:35,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:37,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:45,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:47,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:47,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:18:47,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:18:51,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:52,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:52,349 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:18:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:54,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:18:55,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:18:59,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:01,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:02,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:19:02,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:19:03,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:03,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:07,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:09,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:09,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:11,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:15,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:16,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:16,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:17,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:19:18,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:19:21,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:22,426 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:19:25,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:25,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:27,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:27,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:31,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:31,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:19:33,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:19:33,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:34,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:34,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:35,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:39,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:41,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:41,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:41,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:41,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:41,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:42,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:42,511 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 12:19:42,683 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 12:19:42,759 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 12:19:42,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 12:19:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:43,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:45,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:19:48,881 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:19:49,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:51,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:52,786 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:19:53,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:55,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:56,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:57,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:19:58,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:58,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:19:59,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:00,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:00,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:01,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:03,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:04,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:20:04,113 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:20:05,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:06,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:07,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:08,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:09,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:12,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:13,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:14,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:15,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:16,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:17,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:18,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:19,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:20:19,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:20:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:20,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:22,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:20:22,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:24,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:24,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:26,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:26,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:28,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:29,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:29,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:30,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:31,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:31,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:32,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:33,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:33,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:20:34,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:20:34,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:36,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:42,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:43,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:43,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:48,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:49,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:20:49,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:20:50,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:50,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:52,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:52,938 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:20:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:54,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:56,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:20:59,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:20:59,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:00,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:02,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:02,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:04,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:05,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:21:05,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:21:06,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:06,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:08,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:10,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:11,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:12,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:13,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:16,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:17,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:17,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:18,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:21:20,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:21:22,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:23,013 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:21:24,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:27,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:28,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:30,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:32,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:32,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:34,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:35,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:21:35,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:21:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:36,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:38,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:40,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:42,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:43,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:44,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:47,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:49,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:51,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:21:51,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:21:51,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:53,087 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:21:53,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:55,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:55,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:56,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:57,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:21:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:21:59,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:01,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:03,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:04,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:05,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:06,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:22:06,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:22:06,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:06,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:07,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:09,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:11,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:12,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:12,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:13,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:14,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:14,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:15,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:17,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:19,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:19,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:20,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:22:21,658 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:22:21,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:22,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:22,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,164 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:22:23,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:23,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:25,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:27,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:36,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:22:36,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:22:37,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:41,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:44,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:46,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:47,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:50,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:50,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:51,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:52,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:22:52,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:22:53,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:22:53,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:55,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:57,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:22:59,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:22:59,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:01,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:03,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:03,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:07,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:23:07,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:23:07,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:08,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:08,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:09,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:10,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:10,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:12,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:13,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:15,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:17,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:19,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:21,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:22,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:23:22,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:23:23,310 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:23:23,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:23,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:23,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:27,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:28,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:29,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:29,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:29,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:29,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:30,711 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 12:23:30,908 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 12:23:30,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 12:23:30,976 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 12:23:31,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:31,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:32,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:33,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:35,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:37,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:38,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:23:38,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:23:39,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:39,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:40,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:41,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:43,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:45,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:47,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:49,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:51,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:51,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:53,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:53,430 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:23:53,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:23:53,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:23:53,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:55,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:55,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:56,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:57,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:23:59,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:23:59,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:01,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:01,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:02,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:03,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:04,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:05,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:07,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:08,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:24:08,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:24:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:09,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:10,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:11,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:13,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:14,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:15,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:16,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:16,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:17,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:19,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:19,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:21,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:23,504 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:24:23,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:24:23,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:24:32,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:32,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:32,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:38,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:39,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:24:39,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:24:42,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:42,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:46,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:46,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:50,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:53,578 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:24:54,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:54,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:54,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:24:54,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:24:57,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:58,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:24:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:24:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:01,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:02,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:05,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:09,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:09,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:25:09,794 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:25:10,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:11,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:15,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:15,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:15,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:16,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:17,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:18,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:18,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:19,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:23,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:23,654 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:25:24,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:25,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:25:25,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:25:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:25,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:26,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:26,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:26,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:26,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:27,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:27,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:29,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:33,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:33,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:33,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:35,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:37,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:39,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:25:40,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:25:41,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:41,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:41,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:43,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:44,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:44,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:45,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:47,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:49,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:51,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:52,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:52,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:53,732 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:25:53,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:53,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:54,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:55,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:25:55,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:25:55,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:55,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:57,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:25:58,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:25:59,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:00,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:00,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:01,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:03,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:04,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:04,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:05,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:06,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:06,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:07,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:08,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:08,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:09,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:10,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:26:10,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:26:11,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:11,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:13,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:14,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:15,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:17,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:19,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:19,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:21,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:23,807 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:26:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:25,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:26,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:26:26,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:26:31,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:34,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:37,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:40,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:41,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:26:41,403 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:26:45,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:49,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:51,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:51,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:53,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:26:55,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:26:56,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:26:56,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:26:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:58,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:26:59,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:01,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:01,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:01,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:03,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:04,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:05,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:06,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:06,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:07,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:08,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:11,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:11,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:27:11,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:27:12,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:14,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:16,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:19,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:20,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:22,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:22,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:23,960 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:27:24,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:24,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 12:27:24,873 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 12:27:25,097 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 12:27:25,153 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 12:27:26,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 12:27:26,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:27:27,367 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:27:28,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:30,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:32,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:34,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:36,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:38,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:40,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:42,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:27:42,648 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:27:44,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:46,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:48,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:50,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:52,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:54,051 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:27:54,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:56,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:27:57,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:27:57,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:27:58,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:00,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:02,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:04,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:06,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:08,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:10,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:12,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:13,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:28:13,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:28:14,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:16,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:18,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:20,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:22,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:24,128 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:28:24,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:26,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:28,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:28:28,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:28:30,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:32,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:34,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:36,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:38,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:40,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:43,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:43,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:28:43,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:28:45,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:47,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:49,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:51,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:53,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:54,197 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:28:55,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:57,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:28:59,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:01,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:02,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:29:02,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:29:03,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:05,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:07,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:09,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:11,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:13,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:15,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:17,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:29:17,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:29:19,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:21,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:23,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:24,275 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:29:25,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:27,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:29,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:31,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:33,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:33,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:29:33,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:29:35,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:37,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:39,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:41,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:43,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:45,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:47,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:48,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:29:48,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:29:49,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:51,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:53,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:54,353 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:29:55,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:57,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:29:59,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:01,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:03,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:30:03,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:30:04,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:06,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:08,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:10,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:12,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:14,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:16,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:18,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:30:19,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:30:20,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:22,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:24,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:24,432 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:30:26,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:28,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:30,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:32,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:34,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:35,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:30:35,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:30:36,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:38,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:40,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:42,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:44,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:46,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:48,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:50,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:50,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:30:50,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:30:52,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:54,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:54,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:30:56,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:30:58,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:00,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:02,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:04,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:05,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:31:05,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:31:06,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:08,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:10,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:12,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:14,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:16,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:18,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:20,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:21,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:31:21,518 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:31:22,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:24,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:24,586 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:31:26,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:28,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:31,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:33,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:35,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:36,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:31:36,786 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:31:37,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:39,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:41,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:43,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:45,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:47,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:49,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:51,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:52,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:31:52,094 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:31:53,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:54,664 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:31:55,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:57,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:31:59,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:01,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:03,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:05,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:07,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:32:07,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:32:09,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:11,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:13,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:15,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:17,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:19,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:21,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:32:22,788 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:32:23,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:24,738 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:32:25,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:27,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:29,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:31,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:33,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:35,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:37,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:38,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:32:38,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:32:39,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:41,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:43,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:45,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:47,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:50,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:52,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:32:53,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:32:54,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:54,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:32:56,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:32:58,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:00,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:02,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:04,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:06,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:08,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:33:08,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:33:10,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:12,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:14,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:16,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:18,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:20,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:22,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:23,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:33:23,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:33:24,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:24,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:33:26,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:28,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:30,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:32,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:34,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:36,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:38,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:39,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:33:39,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:33:40,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:42,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:44,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:46,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:48,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:50,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:52,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:54,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:54,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:33:54,590 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:33:54,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:33:56,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:33:58,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:00,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:02,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:05,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:07,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:09,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:09,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:34:09,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:34:11,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:13,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:15,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:17,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:19,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:21,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:23,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:25,022 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:34:25,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:34:25,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:34:25,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:27,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:29,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:31,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:33,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:35,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:37,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:39,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:40,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:34:40,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:34:41,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:43,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:45,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:47,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:49,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:51,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:53,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:55,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:34:55,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:34:55,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:34:57,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:34:59,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:01,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:03,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:05,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:07,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:09,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:10,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:35:10,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:35:11,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:13,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:15,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:17,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:19,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:21,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:23,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:25,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:35:25,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:35:26,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:35:28,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:30,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:32,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:34,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:36,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:38,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:40,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:41,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:35:41,596 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:35:42,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:44,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:46,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:48,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:50,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:52,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:54,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:55,242 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:35:56,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:35:56,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:35:56,913 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:35:58,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:00,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:02,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:04,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:06,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:08,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:10,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:12,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:36:12,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:36:12,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:14,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:16,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:18,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:20,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:22,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:24,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:25,318 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:36:26,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:36:27,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:36:28,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:30,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:32,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:34,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:36,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:38,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:40,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:42,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:36:42,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:36:44,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:46,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:48,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:50,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:52,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:54,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:55,399 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:36:56,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:36:58,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:36:58,125 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:36:58,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:00,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:03,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:05,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:07,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:09,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:11,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:13,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:13,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:37:13,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:37:15,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:17,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:19,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:21,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:23,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:25,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:25,476 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:37:27,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:37:28,806 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:37:29,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:31,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:33,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:35,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:37,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:39,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:41,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:43,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:44,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:37:44,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:37:45,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:47,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:49,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:51,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:53,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:55,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:55,551 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:37:57,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:59,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:37:59,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:37:59,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:38:01,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:03,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:05,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:07,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:09,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:11,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:13,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:14,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:38:14,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:38:15,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:17,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:19,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:21,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:23,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:25,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:25,626 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:38:27,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:29,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:38:29,980 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:38:31,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:33,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:35,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:37,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:39,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:42,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:44,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:38:45,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:38:46,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:48,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:50,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:52,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:54,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:55,706 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:38:56,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:38:58,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:00,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:00,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:39:00,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:39:02,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:04,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:06,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:08,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:10,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:12,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:14,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:39:15,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:39:16,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:18,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:20,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:22,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:24,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:25,781 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:39:26,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:28,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:30,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:31,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:39:31,154 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:39:32,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:34,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:36,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:38,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:40,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:42,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:44,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:46,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:46,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:39:46,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:39:48,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:50,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:52,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:54,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:55,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:39:56,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:39:58,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:00,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:01,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:40:01,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:40:02,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:04,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:06,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:08,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:11,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:13,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:15,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:17,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:40:17,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:40:17,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:19,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:21,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:23,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:25,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:25,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:40:27,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:29,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:31,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:32,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:40:32,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:40:33,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:35,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:37,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:39,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:41,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:43,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:45,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:47,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:40:47,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:40:49,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:51,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:53,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:55,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:56,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:40:57,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:40:59,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:01,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:03,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:41:03,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:41:03,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:05,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:07,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:09,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:11,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:13,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:15,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:17,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:18,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:41:18,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:41:19,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:21,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:23,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:25,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:26,082 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:41:27,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:29,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:31,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:33,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:33,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:41:33,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:41:35,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:37,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:39,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:41,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:43,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:45,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:47,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:49,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:41:49,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:41:50,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:52,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:54,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:56,157 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:41:56,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:41:58,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:00,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:02,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:42:04,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:42:04,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:06,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:08,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:10,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:12,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:14,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:16,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:18,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:20,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:42:20,518 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:42:20,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:22,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:24,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:26,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:42:26,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:28,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:30,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:32,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:34,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:35,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:42:35,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:42:36,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:38,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:40,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:42,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:44,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:46,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:48,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:50,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:51,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:42:51,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:42:52,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:54,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:56,312 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:42:56,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:42:58,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:00,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:02,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:04,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:06,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:43:06,336 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:43:06,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:08,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:10,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:12,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:14,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:16,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:18,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:20,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:21,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:43:21,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:43:22,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:24,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:26,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:43:26,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:28,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:30,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:32,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:34,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:36,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:36,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:43:36,889 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:43:38,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:40,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:43,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:45,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:47,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:49,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:51,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:43:52,206 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:43:53,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:55,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:56,471 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:43:57,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:43:59,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:01,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:03,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:05,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:07,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:44:07,447 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:44:07,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:09,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:11,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:13,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:15,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:17,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:19,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:21,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:22,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:44:22,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:44:23,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:25,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:26,545 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:44:27,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:29,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:31,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:33,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:35,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:37,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:44:37,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:44:39,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:41,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:43,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:45,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:47,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:49,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:51,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:53,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:44:53,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:44:53,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:55,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:56,622 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:44:57,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:44:59,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:01,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:03,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:05,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:07,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:08,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:45:08,579 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:45:09,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:11,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:13,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:15,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:17,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:19,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:21,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:23,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:23,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:45:23,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:45:25,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:26,697 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:45:27,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:29,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:31,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:33,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:35,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:37,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:39,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:45:39,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:45:39,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:41,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:43,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:46,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:48,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:50,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:52,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:54,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:45:54,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:45:54,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:56,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:45:56,774 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:45:58,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:00,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:02,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:04,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:06,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:08,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:09,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:46:09,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:46:10,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:12,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:14,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:16,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:18,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:20,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:22,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:24,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:46:25,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:46:26,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:26,851 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:46:28,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:30,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:32,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:34,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:36,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:38,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:40,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:46:40,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:46:40,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:42,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:44,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:46,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:48,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:50,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:52,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:54,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:55,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:46:55,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:46:56,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:46:56,929 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:46:58,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:00,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:02,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:04,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:06,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:08,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:10,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:10,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:47:10,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:47:13,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:15,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:17,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:19,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:21,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:23,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:25,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:47:26,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:47:27,005 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:47:27,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:29,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:31,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:33,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:35,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:37,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:39,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:47:41,566 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:47:41,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:43,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:45,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:47,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:49,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:51,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:53,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:55,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:56,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:47:56,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:47:57,145 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:47:57,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:47:59,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:01,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:03,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:05,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:07,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:09,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:11,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:12,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:48:12,227 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:48:13,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:15,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:17,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:19,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:21,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:23,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:25,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:27,156 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:48:27,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:48:27,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:48:27,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:29,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:31,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:33,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:35,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:37,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:39,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:41,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:42,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:48:42,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:48:43,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:45,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:47,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:49,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:52,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:54,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:56,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:48:57,230 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:48:58,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:48:58,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:48:58,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:00,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:02,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:04,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:06,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:08,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:10,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:12,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:49:13,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:49:14,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:16,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:18,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:20,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:22,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:24,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:26,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:27,305 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:49:28,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:28,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:49:28,848 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:49:30,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:32,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:34,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:36,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:38,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:40,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:42,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:44,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:49:44,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:49:44,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:46,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:48,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:50,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:52,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:54,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:56,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:57,382 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:49:58,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:49:59,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:49:59,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:50:00,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:02,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:04,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:06,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:08,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:10,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:12,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:50:14,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:50:14,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:16,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:18,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:20,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:22,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:24,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:26,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:27,460 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:50:28,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:50:29,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:50:30,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:32,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:35,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:37,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:39,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:41,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:43,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:45,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:50:45,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:50:45,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:47,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:49,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:51,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:53,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:55,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:57,536 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:50:57,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:50:59,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:51:00,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:51:01,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:03,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:05,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:07,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:09,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:11,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:13,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:51:15,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:51:15,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:17,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:19,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:21,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:23,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:25,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:27,612 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:51:27,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:29,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:30,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:51:30,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:51:31,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:33,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:35,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:37,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:39,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:41,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:43,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:45,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:46,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:51:46,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:51:48,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:50,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:52,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:54,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:56,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:51:57,686 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:51:58,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:00,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:01,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:52:01,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:52:02,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:04,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:06,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:08,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:10,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:12,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:14,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:16,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:16,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:52:16,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:52:18,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:20,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:22,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:24,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:26,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:27,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:52:28,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:30,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:52:32,245 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:52:32,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:34,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:36,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:38,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:40,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:42,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:44,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:46,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:47,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:52:47,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:52:48,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:50,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:52,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:54,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:56,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:52:57,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:52:58,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:00,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:02,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:53:02,889 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:53:04,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:06,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:08,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:10,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:12,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:14,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:16,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:53:18,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:53:18,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:20,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:22,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:24,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:26,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:27,882 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:53:28,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:30,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:33,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:53:33,392 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:53:33,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:35,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:37,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:39,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:41,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:43,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:45,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:47,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:48,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:53:48,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:53:49,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:51,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:53,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:55,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:57,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:53:57,951 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:53:59,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:01,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:03,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:03,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:54:03,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:54:05,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:07,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:09,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:11,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:13,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:15,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:17,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:19,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:54:19,218 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:54:19,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:21,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:23,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:25,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:27,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:28,028 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:54:29,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:31,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:33,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:34,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:54:34,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:54:35,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:38,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:40,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:42,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:44,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:46,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:48,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:49,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:54:49,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:54:50,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:52,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:54,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:56,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:54:58,102 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:54:58,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:00,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:02,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:04,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:05,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:55:05,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:55:06,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:08,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:10,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:12,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:14,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:16,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:18,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:55:20,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:55:20,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:22,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:24,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:26,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:28,167 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:55:28,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:30,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:32,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:34,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:35,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:55:35,607 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:55:36,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:38,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:40,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:42,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:44,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:46,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:48,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:50,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:55:50,896 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:55:50,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:52,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:54,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:56,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:55:58,242 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:55:58,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:00,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:02,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:04,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:56:06,163 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:56:06,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:08,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:11,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:13,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:15,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:17,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:19,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:21,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:56:21,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:56:21,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:23,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:25,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:27,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:28,315 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:56:29,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:31,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:33,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:35,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:36,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:56:36,672 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:56:37,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:39,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:41,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:43,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:45,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:47,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:49,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:56:51,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:56:51,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:53,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:55,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:57,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:56:58,378 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:56:59,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:01,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:03,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:05,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:07,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:57:07,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:57:07,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:09,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:11,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:13,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:15,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:17,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:19,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:21,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:22,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:57:22,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:57:23,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:25,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:27,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:28,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:57:29,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:31,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:33,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:35,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:37,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:57:37,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:57:37,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:39,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:41,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:43,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:45,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:47,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:49,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:51,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:52,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:57:52,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:57:54,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:56,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:57:58,507 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:57:58,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:00,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:02,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:04,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:06,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:58:08,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:58:08,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:10,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:12,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:14,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:16,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:18,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:20,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:22,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:23,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:58:23,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:58:24,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:26,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:28,570 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:58:28,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:30,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:32,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:34,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:36,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:58:38,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:58:38,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:40,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:42,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:44,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:46,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:48,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:50,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:52,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:54,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:58:54,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:58:54,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:56,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:58:58,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:58:58,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:00,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:02,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:04,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:06,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:08,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:09,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:59:09,473 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:59:10,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:13,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:15,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:17,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:19,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:22,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:24,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:24,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:59:24,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:59:26,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:28,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:28,722 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 12:59:30,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:32,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:34,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:36,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:38,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:40,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:40,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:59:40,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:59:42,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:44,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:46,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:48,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:50,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:52,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:54,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:55,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 12:59:55,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 12:59:56,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:58,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 12:59:58,801 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:00:00,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:02,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:04,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:06,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:08,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:10,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:10,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:00:10,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:00:12,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:14,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:16,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:18,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:20,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:22,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:24,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:25,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:00:25,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:00:26,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:28,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:28,877 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:00:30,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:32,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:34,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:36,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:38,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:40,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:41,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:00:41,273 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:00:42,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:44,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:46,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:48,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:50,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:52,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:54,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:56,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:00:56,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:00:57,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:00:58,952 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:00:59,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:01,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:03,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:05,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:07,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:09,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:11,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:12,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:01:12,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:01:13,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:15,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:17,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:19,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:21,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:23,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:25,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:27,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:27,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:01:27,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:01:29,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:01:29,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:31,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:33,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:35,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:37,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:39,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:41,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:42,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:01:42,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:01:43,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:45,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:47,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:49,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:51,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:53,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:55,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:57,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:58,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:01:58,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:01:59,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:01:59,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:02:01,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:03,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:05,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:07,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:09,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:11,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:13,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:13,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:02:13,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:02:15,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:17,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:19,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:21,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:23,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:26,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:28,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:28,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:02:28,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:02:29,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:02:30,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:32,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:34,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:36,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:38,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:40,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:42,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:43,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:02:43,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:02:44,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:46,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:48,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:50,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:52,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:54,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:56,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:58,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:02:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:02:59,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:02:59,466 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:03:00,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:02,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:04,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:06,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:08,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:10,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:12,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:14,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:14,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:03:14,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:03:16,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:18,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:20,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:22,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:24,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:26,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:28,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:29,322 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:03:29,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:03:29,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:03:30,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:32,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:34,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:36,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:38,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:40,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:42,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:44,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:44,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:03:44,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:03:46,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:48,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:50,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:52,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:54,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:56,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:58,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:03:59,398 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:04:00,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:00,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:04:00,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:04:02,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:04,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:06,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:08,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:10,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:12,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:14,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:15,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:04:15,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:04:16,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:18,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:20,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:22,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:24,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:26,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:29,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:29,472 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:04:30,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:04:30,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:04:31,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:33,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:35,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:37,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:39,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:41,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:43,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:45,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:04:46,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:04:47,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:49,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:51,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:53,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:55,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:57,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:59,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:04:59,549 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:05:01,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:01,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:05:01,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:05:03,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:05,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:07,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:09,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:11,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:13,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:15,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:16,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:05:16,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:05:17,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:19,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:21,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:23,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:25,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:27,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:29,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:29,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:05:31,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:05:32,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:05:33,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:35,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:37,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:39,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:41,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:43,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:45,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:47,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:05:47,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:05:48,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:50,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:52,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:54,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:56,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:58,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:05:59,699 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:06:00,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:02,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:02,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:06:02,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:06:04,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:06,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:08,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:10,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:12,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:14,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:16,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:17,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:06:17,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:06:18,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:20,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:22,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:24,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:26,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:28,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:29,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:06:30,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:32,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:33,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:06:33,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:06:34,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:36,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:38,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:40,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:42,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:45,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:47,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:48,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:06:48,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:06:49,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:51,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:53,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:55,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:57,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:59,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:06:59,849 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:07:01,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:03,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:03,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:07:03,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:07:05,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:07,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:09,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:11,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:13,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:15,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:17,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:18,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:07:18,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:07:19,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:21,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:23,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:25,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:27,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:29,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:29,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:07:31,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:33,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:34,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:07:34,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:07:35,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:37,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:39,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:41,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:43,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:45,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:47,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:49,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:49,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:07:49,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:07:51,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:53,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:55,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:57,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:07:59,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:00,003 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:08:01,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:03,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:04,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:08:04,829 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:08:05,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:07,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:09,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:11,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:13,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:15,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:17,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:19,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:08:20,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:08:22,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:24,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:26,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:28,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:30,082 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:08:30,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:32,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:34,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:08:35,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:08:36,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:38,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:40,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:42,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:44,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:46,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:48,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:50,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:50,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:08:50,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:08:52,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:54,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:56,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:08:58,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:00,157 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:09:00,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:02,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:04,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:09:05,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:09:06,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:08,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:10,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:12,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:14,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:16,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:09:21,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:09:21,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:09:30,233 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:09:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:09:36,343 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:09:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:09:51,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:10:00,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:10:07,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:10:07,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:10:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:10:13,957 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 13:10:14,007 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 13:10:14,085 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 13:10:14,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 13:10:15,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:10:16,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:10:20,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:10:22,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:10:22,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:10:30,394 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:10:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:10:37,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:10:38,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:10:52,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:10:52,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:11:00,473 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:11:08,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:11:08,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:11:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:11:23,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:11:30,542 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:11:38,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:11:38,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:11:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:11:53,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:12:00,619 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:12:08,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:12:08,953 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:12:24,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:12:24,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:12:30,696 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:12:39,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:12:39,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:12:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:12:54,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:13:00,776 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:13:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:13:09,881 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:13:25,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:13:25,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:13:30,853 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:13:40,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:13:40,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:13:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:13:55,740 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:14:00,929 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:14:10,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:14:10,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:14:26,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:14:26,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:14:31,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:14:31,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:34,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:35,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:37,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:39,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:41,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:41,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:14:41,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:14:43,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:43,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:47,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:48,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:49,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:51,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:51,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:53,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:55,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:56,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:14:56,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:14:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:56,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:14:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:57,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:14:59,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:00,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:00,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,080 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:15:01,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:03,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:05,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:06,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:07,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:09,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:10,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:10,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:12,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:15:12,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:15:13,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:13,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:13,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:14,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:15,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:17,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:18,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:18,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:19,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:21,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:23,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:24,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:15:27,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:15:28,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:28,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:31,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:15:31,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:33,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:35,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:37,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:37,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:39,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:41,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:42,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:15:42,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:15:43,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:46,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:46,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:47,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:51,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:15:53,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:15:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:15:57,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:15:58,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:00,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:01,215 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:16:04,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:06,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:06,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:06,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:08,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:11,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:11,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:12,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:16:12,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:16:14,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:16,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:16,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:19,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:19,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:20,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:20,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:21,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:21,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:22,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:23,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:24,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:26,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:27,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:27,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:28,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:16:28,218 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:16:28,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:29,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:29,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:31,286 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:16:32,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:34,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:36,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:37,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:38,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:38,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:40,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:41,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:42,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:43,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:16:43,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:16:44,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:45,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:46,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:48,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:50,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:51,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:52,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:53,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:53,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:54,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:54,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:55,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:56,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:58,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:16:58,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:16:58,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:16:58,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:17:00,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:01,344 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:17:01,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:01,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:02,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:04,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:05,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:06,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:08,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:09,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:10,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:12,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:17:14,206 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:17:14,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:16,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:19,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:24,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:27,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:27,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:17:29,473 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:17:29,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:29,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:30,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:31,416 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:17:33,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:33,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:34,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:38,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:39,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:42,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:44,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:44,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:17:44,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:17:46,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:46,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:48,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:49,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:49,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:52,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:54,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:54,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:54,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:55,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:17:58,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:17:59,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:17:59,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:18:00,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:01,495 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:18:03,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:03,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:05,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:07,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:07,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:08,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:08,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:09,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:10,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:11,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:12,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:13,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:14,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:15,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:18:15,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:18:15,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:17,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:17,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:17,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:18,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:18,251 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 13:18:18,458 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 13:18:18,529 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 13:18:19,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 13:18:19,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:20,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:21,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:25,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:25,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:27,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:29,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:29,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:30,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:18:30,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:18:31,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:31,581 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:18:33,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:35,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:35,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:37,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:39,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:39,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:41,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:43,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:44,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:44,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:45,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:45,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:18:45,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:18:47,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:48,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:49,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:51,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:53,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:53,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:55,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:56,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:56,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:57,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:18:59,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:18:59,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:01,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:19:01,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:19:01,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:01,653 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:19:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:02,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:03,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:04,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:04,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:05,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:07,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:08,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:09,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:11,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:16,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:19:16,457 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:19:19,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:20,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:25,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:28,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:29,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:29,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:19:31,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:19:31,926 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:19:33,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:35,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:37,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:38,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:41,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:41,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:45,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:46,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:19:46,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:19:47,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:51,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:51,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:54,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:54,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:19:55,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:19:57,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:01,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:01,795 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:20:02,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:20:02,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:20:03,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:03,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:05,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:08,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:08,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:11,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:14,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:14,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:15,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:17,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:20:17,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:20:17,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:18,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:18,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:19,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:21,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:22,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:22,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:23,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:25,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:25,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:27,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:29,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:31,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:31,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:20:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:20:32,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:20:33,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:35,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:37,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:37,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:39,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:41,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:42,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:42,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:44,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:46,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:47,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:48,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:20:48,201 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:20:48,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:50,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:50,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:51,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:52,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:53,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:53,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:54,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:55,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:56,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:58,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:20:58,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:20:58,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:00,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:00,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:01,931 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:21:02,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:03,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:21:03,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:21:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:03,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:04,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:06,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:08,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:08,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:14,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:16,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:16,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:18,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:18,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:21:18,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:21:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:22,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:26,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:30,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:32,004 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:21:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:21:33,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:21:34,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:35,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:35,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:37,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:38,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:38,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:40,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:41,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:43,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:44,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:46,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:48,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:21:49,163 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:21:49,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:49,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:50,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:51,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:21:52,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:54,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:56,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:21:58,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:00,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:00,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:02,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:22:02,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:04,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:22:04,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:22:04,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:04,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:04,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:06,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:06,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:07,652 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 13:22:07,818 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 13:22:07,892 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 13:22:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:08,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 13:22:08,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:09,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:09,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:12,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:14,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:15,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:16,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:18,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:18,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:18,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:19,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:22:19,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:22:20,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:21,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:21,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:22,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:23,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:24,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:26,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:28,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:30,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:32,160 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:22:32,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:34,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:22:34,856 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:22:36,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:36,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:38,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:40,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:40,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:41,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:42,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:43,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:43,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:44,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:46,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:48,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:48,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:48,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:49,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:22:50,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:22:50,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:52,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:54,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:56,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:22:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:57,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:22:58,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:00,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:02,223 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:23:02,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:05,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:23:05,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:23:10,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:16,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:18,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:18,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:23:20,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:23:20,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:24,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:25,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:28,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:32,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:23:32,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:35,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:35,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:23:35,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:23:39,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:42,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:42,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:43,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:45,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:45,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:47,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:48,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:49,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:51,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:23:51,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:23:51,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:55,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:57,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:23:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:23:59,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:01,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:02,351 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:24:04,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:04,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:04,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:04,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:05,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:06,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:24:06,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:24:07,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:09,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:10,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:10,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:11,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:12,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:13,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:15,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:15,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:18,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:19,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:19,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:21,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:21,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:21,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:24:21,978 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:24:22,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:22,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:23,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:23,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:25,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:26,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:26,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:27,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:29,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:29,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:31,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:32,415 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:24:32,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:32,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:32,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:33,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:35,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:36,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:37,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:24:37,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:24:37,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:39,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:39,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:40,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:40,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:41,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:42,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:42,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:43,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:45,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:47,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:48,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:48,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:51,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:52,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:24:52,600 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:24:53,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:55,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:24:55,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:24:57,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:00,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:02,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:25:07,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:25:07,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:25:07,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:09,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:09,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:11,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:12,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:12,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:15,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:19,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:20,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:20,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:25:23,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:25:23,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:23,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:27,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:28,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:29,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:31,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:31,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:32,541 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:25:33,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:33,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:37,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:25:38,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:25:39,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:41,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:43,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:45,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:46,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:46,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:47,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:51,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:53,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:54,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:25:54,099 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:25:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:54,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:55,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:56,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:56,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:57,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:25:59,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:25:59,875 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 13:26:00,023 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 13:26:00,077 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 13:26:00,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 13:26:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:01,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:02,632 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:26:02,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:02,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:03,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:05,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:05,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:07,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:09,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:26:09,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:26:09,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:11,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:13,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:14,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:14,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:16,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:18,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:18,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:19,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:20,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:22,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:26:24,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:26:24,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:25,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:26,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:28,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:30,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:32,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:26:32,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:34,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:36,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:36,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:38,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:38,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:39,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:26:39,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:26:40,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:42,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:43,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:44,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:45,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:26:46,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:48,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:50,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:53,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:26:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:26:55,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:27:01,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:02,774 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:27:04,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:04,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:05,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:06,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:06,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:09,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:10,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:27:10,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:27:13,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:16,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:17,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:18,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:21,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:23,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:24,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:25,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:27:25,565 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:27:26,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:26,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:27,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:29,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:31,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:31,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:32,846 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:27:33,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:37,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:38,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:38,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:39,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:40,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:27:40,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:27:42,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:42,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:43,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:44,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:44,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:45,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:47,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:51,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:52,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:52,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:52,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:52,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:53,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:55,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:56,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:27:56,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:27:56,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:56,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:57,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:59,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:27:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:27:59,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:00,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:01,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:02,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:28:03,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:05,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:05,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:05,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:07,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:08,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:08,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:09,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:11,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:28:11,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:28:13,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:13,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:13,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:15,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:15,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:17,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:19,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:20,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:21,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:21,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:22,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:23,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:25,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:26,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:28:26,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:28:27,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:29,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:29,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:30,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:31,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:32,993 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:28:33,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:34,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:34,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:35,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:37,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:39,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:41,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:41,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:28:42,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:28:42,137 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:28:43,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:45,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:56,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:28:57,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:28:57,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:29:00,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:03,069 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:29:04,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:04,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:06,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:10,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:10,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:12,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:29:12,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:29:14,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:16,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:19,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:20,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:24,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:26,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:27,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:29:27,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:29:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:30,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:32,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:32,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:29:33,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:33,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:34,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:35,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:36,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:38,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:38,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:42,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:29:43,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:29:44,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:45,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:45,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:48,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:49,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:50,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:50,976 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 13:29:51,172 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 13:29:51,232 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 13:29:52,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 13:29:52,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:52,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:52,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:53,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:54,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:54,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:56,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:29:58,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:29:58,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:29:58,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:30:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:00,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:02,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:03,280 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:30:04,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:04,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:05,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:06,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:06,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:08,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:10,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:11,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:11,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:13,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:13,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:30:13,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:30:14,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:14,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:15,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:15,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:17,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:17,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:19,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:20,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:20,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:22,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:23,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:25,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:27,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:29,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:30:29,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:30:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:31,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:32,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:32,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:33,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:33,349 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:30:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:34,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:35,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:39,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:40,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:40,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:41,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:42,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:43,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:44,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:30:44,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:30:45,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:53,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:30:54,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:54,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:30:57,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:00,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:31:00,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:31:01,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:03,420 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:31:05,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:06,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:08,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:09,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:13,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:15,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:31:15,343 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:31:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:17,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:19,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:23,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:25,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:27,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:29,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:31:30,573 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:31:31,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:33,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:33,493 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:31:34,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:34,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:37,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:39,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:41,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:44,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:45,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:45,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:31:45,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:31:47,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:49,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:49,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:49,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:51,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:53,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:54,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:54,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:55,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:56,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:56,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:31:57,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:31:57,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:01,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:32:01,127 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:32:01,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:02,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:03,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:32:03,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:05,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:05,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:07,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:09,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:11,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:13,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:15,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:15,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:16,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:32:16,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:32:16,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:16,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:17,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:19,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:20,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:21,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:23,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:24,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:24,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:25,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:27,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:27,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:28,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:29,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:31,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:32:31,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:32:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:31,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:33,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:33,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:32:34,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:35,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:37,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:43,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:44,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:44,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:46,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:32:46,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:32:47,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:49,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:53,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:57,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:32:58,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:32:58,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:00,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:02,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:33:02,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:33:02,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:03,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:33:04,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:08,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:12,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:13,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:14,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:33:17,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:33:17,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:17,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:18,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:20,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:22,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:23,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:26,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:28,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:28,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:28,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:30,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:31,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:32,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:33:32,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:33:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:33,792 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:33:34,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:35,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:35,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:35,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:35,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:36,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 13:33:36,379 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 13:33:36,585 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 13:33:36,646 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 13:33:37,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 13:33:37,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:38,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:40,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:42,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:44,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:46,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:48,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:33:48,002 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:33:48,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:50,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:52,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:54,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:56,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:33:58,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:00,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:02,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:03,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:34:03,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:34:04,102 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:34:04,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:06,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:08,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:10,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:12,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:14,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:17,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:34:19,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:34:19,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:21,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:23,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:25,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:27,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:29,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:31,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:33,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:34,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:34:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:34:34,416 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:34:35,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:37,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:39,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:41,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:43,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:45,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:47,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:49,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:34:49,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:34:51,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:53,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:55,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:57,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:34:59,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:01,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:03,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:04,079 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:35:04,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:35:04,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:35:05,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:07,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:09,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:11,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:13,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:15,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:17,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:19,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:20,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:35:20,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:35:21,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:23,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:25,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:27,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:29,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:31,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:33,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:34,154 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:35:35,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:35,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:35:35,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:35:37,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:39,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:41,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:43,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:45,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:47,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:49,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:50,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:35:50,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:35:51,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:53,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:56,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:35:58,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:00,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:02,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:04,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:36:04,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:06,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:36:06,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:36:06,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:08,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:10,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:12,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:14,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:16,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:18,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:20,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:21,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:36:21,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:36:22,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:24,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:26,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:28,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:30,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:32,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:34,306 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:36:34,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:36,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:36,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:36:36,632 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:36:38,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:40,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:42,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:44,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:46,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:48,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:50,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:51,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:36:51,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:36:52,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:54,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:56,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:36:58,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:00,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:02,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:04,379 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:37:04,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:06,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:07,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:37:07,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:37:08,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:10,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:12,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:14,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:16,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:18,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:20,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:22,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:22,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:37:22,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:37:25,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:27,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:29,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:31,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:33,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:34,457 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:37:35,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:37,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:37:37,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:37:39,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:41,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:43,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:45,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:47,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:49,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:51,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:37:52,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:37:53,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:55,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:57,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:37:59,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:01,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:03,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:04,531 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:38:05,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:07,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:38:08,143 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:38:09,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:11,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:13,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:15,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:17,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:19,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:21,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:23,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:23,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:38:23,582 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:38:25,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:27,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:29,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:31,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:33,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:34,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:38:35,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:37,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:38,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:38:38,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:38:39,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:41,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:43,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:45,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:47,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:50,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:52,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:38:54,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:38:54,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:56,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:38:58,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:00,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:02,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:04,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:04,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:39:06,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:08,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:09,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:39:09,575 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:39:10,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:12,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:14,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:16,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:18,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:20,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:22,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:24,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:39:24,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:39:26,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:28,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:30,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:32,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:34,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:34,759 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:39:36,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:38,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:40,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:39:40,156 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:39:40,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:42,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:44,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:46,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:48,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:50,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:52,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:54,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:39:55,491 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:39:56,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:39:58,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:00,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:02,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:04,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:04,831 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:40:06,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:08,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:10,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:40:10,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:40:12,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:14,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:16,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:18,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:20,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:22,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:24,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:26,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:40:26,020 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:40:26,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:28,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:30,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:32,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:34,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:34,907 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:40:36,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:38,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:40,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:41,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:40:41,307 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:40:42,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:44,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:47,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:49,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:51,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:53,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:55,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:56,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:40:56,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:40:57,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:40:59,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:01,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:03,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:04,978 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:41:05,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:07,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:09,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:11,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:11,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:41:11,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:41:13,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:15,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:17,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:19,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:21,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:23,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:25,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:41:27,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:41:27,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:29,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:31,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:33,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:35,048 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:41:35,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:37,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:39,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:41,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:42,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:41:42,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:41:43,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:45,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:47,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:49,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:51,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:53,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:55,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:41:57,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:41:57,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:41:58,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:00,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:02,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:04,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:05,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:42:06,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:08,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:10,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:12,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:13,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:42:13,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:42:14,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:16,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:18,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:20,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:22,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:24,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:26,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:42:28,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:42:28,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:30,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:32,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:34,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:35,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:42:36,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:38,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:40,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:42,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:43,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:42:43,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:42:44,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:46,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:48,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:50,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:52,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:54,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:56,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:42:58,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:42:58,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:42:59,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:01,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:03,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:05,257 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:43:05,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:07,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:09,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:11,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:13,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:14,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:43:14,028 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:43:15,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:17,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:19,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:21,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:23,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:25,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:27,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:29,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:43:29,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:43:29,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:31,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:33,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:35,332 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:43:35,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:37,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:39,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:41,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:43,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:44,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:43:44,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:43:45,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:47,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:49,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:51,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:53,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:55,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:57,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:43:59,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:00,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:44:00,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:44:01,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:03,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:05,408 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:44:05,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:07,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:09,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:11,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:13,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:44:15,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:44:15,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:17,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:19,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:21,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:23,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:25,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:27,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:29,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:44:30,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:44:31,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:33,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:35,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:44:35,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:37,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:39,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:41,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:43,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:45,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:44:46,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:44:47,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:49,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:52,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:54,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:56,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:44:58,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:00,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:01,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:45:01,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:45:02,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:04,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:05,559 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:45:06,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:08,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:10,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:12,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:14,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:16,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:16,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:45:16,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:45:18,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:20,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:22,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:24,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:26,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:28,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:30,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:32,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:45:32,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:45:32,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:34,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:35,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:45:36,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:38,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:40,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:42,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:44,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:46,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:47,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:45:47,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:45:48,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:51,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:53,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:55,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:57,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:45:59,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:01,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:02,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:46:02,553 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:46:03,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:05,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:05,703 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:46:07,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:09,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:11,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:13,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:15,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:17,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:17,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:46:17,902 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:46:19,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:21,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:23,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:25,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:27,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:29,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:31,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:33,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:46:33,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:46:33,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:35,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:35,778 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:46:37,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:39,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:41,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:43,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:45,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:47,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:46:48,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:46:49,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:51,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:53,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:55,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:57,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:46:59,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:01,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:03,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:03,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:47:03,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:47:05,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:05,852 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:47:07,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:10,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:12,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:14,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:16,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:18,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:47:18,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:47:20,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:22,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:24,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:26,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:28,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:30,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:32,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:34,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:47:34,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:47:34,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:35,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:47:36,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:38,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:40,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:42,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:44,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:46,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:48,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:49,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:47:49,568 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:47:50,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:52,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:54,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:56,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:47:58,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:00,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:02,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:04,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:48:04,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:48:06,004 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:48:06,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:08,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:10,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:12,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:14,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:16,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:19,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:48:20,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:48:21,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:23,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:25,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:27,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:29,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:31,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:33,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:35,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:48:35,567 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:48:35,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:36,078 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:48:37,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:39,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:41,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:43,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:45,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:47,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:49,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:50,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:48:50,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:48:51,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:53,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:55,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:57,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:48:59,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:01,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:03,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:05,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:06,149 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:49:06,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:49:06,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:49:07,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:09,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:11,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:13,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:15,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:17,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:19,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:21,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:49:21,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:49:21,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:23,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:25,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:27,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:29,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:31,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:33,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:35,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:36,208 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:49:36,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:49:36,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:49:37,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:39,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:42,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:44,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:46,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:48,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:50,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:51,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:49:51,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:49:52,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:54,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:56,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:49:58,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:00,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:02,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:04,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:06,286 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:50:06,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:50:07,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:50:08,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:10,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:12,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:14,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:16,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:18,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:20,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:50:22,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:50:22,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:24,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:26,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:28,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:30,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:32,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:34,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:36,364 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:50:36,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:37,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:50:37,848 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:50:38,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:40,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:42,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:44,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:46,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:48,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:50,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:52,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:50:53,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:50:55,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:57,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:50:59,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:01,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:03,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:05,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:06,441 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:51:07,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:51:08,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:51:09,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:11,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:13,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:15,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:17,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:19,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:21,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:23,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:51:23,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:51:23,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:25,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:27,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:29,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:31,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:33,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:35,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:36,517 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:51:37,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:38,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:51:38,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:51:39,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:41,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:43,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:45,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:47,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:49,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:51,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:53,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:54,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:51:54,125 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:51:55,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:57,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:51:59,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:01,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:03,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:05,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:06,589 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:52:07,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:52:09,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:52:09,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:11,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:13,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:15,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:17,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:19,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:21,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:24,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:52:24,582 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:52:24,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:26,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:28,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:30,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:32,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:34,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:36,662 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:52:36,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:38,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:39,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:52:39,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:52:40,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:42,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:44,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:46,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:48,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:50,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:52,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:54,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:52:55,129 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:52:56,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:52:58,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:00,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:02,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:04,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:06,735 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:53:06,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:08,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:10,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:53:10,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:53:10,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:12,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:14,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:16,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:18,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:20,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:22,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:24,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:25,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:53:25,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:53:26,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:28,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:30,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:32,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:34,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:36,807 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:53:36,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:38,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:40,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:41,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:53:41,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:53:42,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:44,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:46,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:48,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:50,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:52,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:54,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:56,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:53:56,294 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:53:57,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:53:59,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:01,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:03,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:05,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:06,888 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:54:07,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:09,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:11,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:54:11,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:54:11,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:13,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:15,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:17,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:19,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:21,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:23,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:25,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:54:26,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:54:27,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:29,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:31,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:33,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:35,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:36,957 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:54:37,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:39,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:41,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:54:42,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:54:43,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:45,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:47,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:49,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:51,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:53,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:55,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:57,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:54:57,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:54:57,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:54:59,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:01,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:03,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:05,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:07,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:55:07,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:09,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:11,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:12,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:55:12,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:55:13,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:15,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:17,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:19,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:21,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:23,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:25,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:27,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:55:27,805 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:55:27,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:29,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:31,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:34,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:36,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:37,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:55:38,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:40,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:42,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:55:43,096 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:55:44,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:46,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:48,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:50,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:52,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:54,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:56,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:55:58,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:55:58,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:55:58,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:00,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:02,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:04,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:06,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:07,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:56:08,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:10,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:12,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:13,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:56:13,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:56:14,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:16,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:18,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:20,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:22,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:24,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:26,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:28,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:29,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:56:29,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:56:30,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:32,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:34,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:36,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:37,249 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:56:38,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:40,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:42,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:44,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:56:44,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:56:44,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:46,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:48,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:50,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:52,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:54,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:56,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:58,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:56:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:56:59,731 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:57:00,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:02,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:05,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:07,325 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:57:07,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:09,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:11,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:13,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:15,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:57:15,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:57:15,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:17,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:19,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:21,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:23,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:25,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:27,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:29,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:57:30,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:57:31,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:33,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:35,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:37,401 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:57:37,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:39,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:41,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:43,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:45,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:57:45,594 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:57:45,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:47,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:49,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:51,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:53,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:55,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:57,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:57:59,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:00,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:58:00,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:58:01,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:03,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:05,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:07,473 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:58:07,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:09,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:11,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:13,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:15,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:58:16,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:58:17,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:19,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:21,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:23,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:26,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:28,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:30,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:58:31,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:58:32,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:34,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:36,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:37,550 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:58:38,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:40,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:42,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:44,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:46,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:58:46,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:58:46,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:48,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:50,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:52,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:54,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:56,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:58:58,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:00,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:01,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:59:01,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:59:02,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:04,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:06,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:07,624 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:59:08,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:10,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:12,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:14,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:16,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:59:17,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:59:18,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:20,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:22,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:24,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:26,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:28,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:30,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:32,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:59:32,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:59:32,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:34,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:36,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:37,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 13:59:38,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:40,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:42,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:44,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:46,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:47,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 13:59:47,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 13:59:48,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:50,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:52,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:54,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:56,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 13:59:59,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:01,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:03,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:00:03,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:00:03,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:05,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:07,771 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:00:07,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:09,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:11,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:13,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:15,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:17,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:00:18,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:00:19,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:21,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:23,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:25,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:27,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:29,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:31,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:33,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:00:33,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:00:33,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:35,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:37,857 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:00:38,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:40,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:42,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:44,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:46,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:48,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:50,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:52,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:00:52,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:00:52,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:54,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:56,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:00:58,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:00,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:02,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:04,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:06,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:07,925 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:01:08,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:01:08,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:01:08,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:10,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:12,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:14,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:16,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:18,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:20,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:22,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:24,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:01:24,006 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:01:24,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:26,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:28,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:30,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:32,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:34,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:36,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:37,999 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:01:38,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:39,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:01:39,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:01:40,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:42,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:44,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:46,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:48,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:51,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:53,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:54,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:01:54,626 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:01:55,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:57,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:01:59,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:01,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:03,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:05,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:07,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:08,074 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:02:09,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:09,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:02:09,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:02:11,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:13,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:15,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:17,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:19,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:21,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:24,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:25,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:02:25,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:02:26,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:28,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:30,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:32,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:34,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:36,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:38,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:38,148 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:02:40,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:40,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:02:40,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:02:42,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:44,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:46,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:48,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:50,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:52,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:54,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:55,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:02:55,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:02:56,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:02:58,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:00,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:02,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:04,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:06,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:08,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:08,224 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:03:10,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:11,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:03:11,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:03:12,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:14,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:16,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:18,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:20,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:22,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:24,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:26,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:03:26,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:03:28,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:30,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:32,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:34,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:36,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:38,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:38,298 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:03:40,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:41,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:03:41,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:03:42,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:44,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:46,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:48,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:50,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:52,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:54,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:56,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:03:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:03:56,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:03:58,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:00,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:02,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:04,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:06,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:08,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:08,378 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:04:10,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:12,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:12,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:04:12,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:04:14,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:17,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:19,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:21,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:23,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:25,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:27,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:27,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:04:27,517 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:04:29,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:31,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:33,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:35,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:37,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:38,459 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:04:39,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:41,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:42,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:04:42,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:04:43,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:45,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:47,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:49,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:51,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:53,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:55,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:57,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:04:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:04:58,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:04:59,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:01,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:03,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:05,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:07,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:08,533 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:05:09,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:11,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:13,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:13,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:05:13,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:05:15,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:17,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:19,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:21,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:23,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:25,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:27,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:05:28,944 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:05:29,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:31,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:33,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:36,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:38,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:38,609 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:05:40,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:42,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:44,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:05:44,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:05:46,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:48,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:50,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:52,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:54,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:56,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:58,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:05:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:05:59,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:06:00,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:02,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:04,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:06,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:08,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:08,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:06:10,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:12,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:14,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:15,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:06:15,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:06:16,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:18,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:20,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:22,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:24,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:26,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:28,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:30,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:30,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:06:30,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:06:32,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:34,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:36,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:38,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:38,761 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:06:40,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:42,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:44,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:45,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:06:45,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:06:46,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:48,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:50,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:53,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:55,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:57,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:06:59,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:00,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:07:00,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:07:01,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:03,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:05,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:07,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:08,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:07:09,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:11,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:13,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:15,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:16,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:07:16,059 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:07:17,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:19,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:21,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:23,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:25,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:27,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:29,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:31,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:07:31,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:07:33,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:35,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:37,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:38,914 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:07:39,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:41,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:43,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:45,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:46,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:07:46,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:07:47,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:49,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:51,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:53,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:55,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:57,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:07:59,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:01,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:02,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:08:02,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:08:03,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:05,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:08,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:08,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:08:10,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:12,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:14,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:16,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:08:17,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:08:18,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:20,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:22,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:24,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:26,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:28,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:30,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:32,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:32,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:08:32,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:08:34,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:36,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:38,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:39,050 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:08:40,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:42,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:44,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:46,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:47,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:08:47,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:08:48,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:50,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:52,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:54,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:56,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:08:58,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:00,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:02,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:09:03,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:09:04,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:06,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:08,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:09,125 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:09:10,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:12,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:14,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:16,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:18,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:18,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:09:18,566 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:09:20,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:22,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:24,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:26,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:28,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:30,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:32,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:33,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:09:33,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:09:34,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:36,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:38,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:39,199 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:09:40,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:42,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:44,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:46,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:48,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:49,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:09:49,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:09:50,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:52,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:54,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:56,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:09:59,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:01,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:03,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:04,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:10:04,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:10:05,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:07,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:09,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:09,273 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:10:11,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:13,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:15,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:17,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:19,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:19,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:10:19,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:10:21,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:23,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:25,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:27,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:29,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:31,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:33,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:10:35,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:10:35,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:37,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:39,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:39,351 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:10:41,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:43,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:45,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:47,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:49,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:50,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:10:50,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:10:51,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:53,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:55,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:57,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:10:59,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:01,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:03,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:05,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:05,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:11:05,748 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:11:07,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:09,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:09,423 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:11:11,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:13,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:15,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:17,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:19,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:11:21,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:11:21,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:23,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:25,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:27,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:29,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:31,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:33,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:35,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:11:36,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:11:37,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:39,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:39,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:11:41,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:44,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:46,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:48,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:50,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:51,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:11:51,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:11:52,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:54,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:56,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:11:58,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:00,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:02,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:04,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:06,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:08,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:12:08,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:12:08,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:09,570 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:12:10,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:12,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:14,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:16,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:18,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:20,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:22,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:23,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:12:23,442 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:12:24,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:26,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:28,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:30,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:32,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:34,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:36,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:38,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:38,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:12:38,677 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:12:39,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:12:40,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:42,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:44,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:46,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:48,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:50,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:52,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:12:53,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:12:54,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:56,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:12:58,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:00,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:02,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:04,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:06,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:08,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:09,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:13:09,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:13:09,720 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:13:10,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:12,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:14,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:16,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:19,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:21,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:23,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:24,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:13:24,421 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:13:25,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:27,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:29,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:31,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:33,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:35,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:37,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:39,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:13:39,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:13:40,002 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:13:41,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:43,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:45,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:47,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:49,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:51,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:53,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:55,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:13:55,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:13:55,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:57,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:13:59,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:01,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:03,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:05,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:07,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:09,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:09,868 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:14:10,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:14:10,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:14:11,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:13,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:15,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:17,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:19,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:21,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:23,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:25,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:25,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:14:25,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:14:27,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:29,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:31,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:33,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:35,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:37,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:39,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:39,945 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:14:40,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:14:40,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:14:41,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:43,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:45,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:47,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:49,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:51,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:53,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:55,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:56,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:14:56,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:14:57,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:14:59,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:01,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:03,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:06,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:08,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:10,022 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:15:10,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:15:11,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:15:12,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:14,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:16,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:18,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:20,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:22,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:24,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:26,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:26,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:15:26,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:15:28,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:30,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:32,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:34,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:36,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:38,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:40,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:15:40,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:41,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:15:41,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:15:42,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:44,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:46,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:15:57,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:15:57,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:16:10,174 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:16:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:16:12,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:16:27,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:16:27,740 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:16:40,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:16:43,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:16:43,020 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:16:44,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:16:44,619 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 14:16:44,671 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 14:16:44,728 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 14:16:45,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 14:16:46,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:16:47,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:16:50,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:16:58,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:16:58,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:17:06,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:17:10,334 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:17:13,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:17:13,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:17:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:17:28,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:17:40,419 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:17:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:17:44,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:17:59,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:17:59,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:18:10,499 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:18:14,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:18:14,801 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:18:30,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:18:30,029 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:18:40,575 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:18:45,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:18:45,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:19:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:19:00,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:19:10,648 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:19:15,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:19:15,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:19:31,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:19:31,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:19:40,723 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:19:46,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:19:46,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:20:01,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:20:01,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:20:10,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:20:16,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:20:16,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:20:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:20:32,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:20:40,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:20:47,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:20:47,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:21:02,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:02,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:21:02,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:21:03,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:03,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:05,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:07,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:09,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:09,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:10,958 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:21:11,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:11,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:11,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:13,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:15,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:16,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:16,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:17,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:17,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:18,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:21:18,024 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:21:19,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:21,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:23,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:25,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:26,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:26,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:27,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:29,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:31,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:33,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:33,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:21:33,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:21:33,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:35,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:35,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:37,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:38,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:39,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:41,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:21:41,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:41,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:43,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:45,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:47,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:48,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:21:48,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:21:49,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:49,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:21:51,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:21:53,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:02,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:02,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:03,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:03,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:22:03,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:22:07,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:09,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:09,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:11,100 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:22:11,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:15,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:15,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:15,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:17,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:19,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:22:19,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:22:19,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:20,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:20,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:23,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:27,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:29,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:29,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:33,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:34,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:22:34,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:22:35,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:36,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:36,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:38,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:39,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:41,170 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:22:41,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:45,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:45,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:47,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:49,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:49,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:22:49,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:22:49,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:49,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:51,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:53,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:55,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:56,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:57,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:22:58,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:22:58,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:00,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:01,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:03,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:04,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:04,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:23:04,963 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:23:05,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:07,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:08,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:09,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:11,242 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:23:11,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:11,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:13,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:15,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:16,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:17,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:19,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:20,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:23:20,327 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:23:21,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:21,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:23,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:23,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:23,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:24,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:25,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:26,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:27,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:29,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:31,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:31,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:32,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:32,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:33,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:34,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:35,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:35,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:23:35,589 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:23:37,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:37,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:38,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:38,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:39,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:40,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:40,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,315 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:23:41,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:41,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:42,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:42,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:43,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:45,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:47,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:47,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:47,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:48,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:23:50,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:23:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:23:50,848 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:24:00,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:04,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:06,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:24:06,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:24:08,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:08,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:10,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:10,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:11,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:24:12,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:16,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:19,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:20,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:24:21,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:24:22,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:22,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:23,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:26,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:30,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:30,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:32,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:36,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:37,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:24:37,029 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:24:38,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:38,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:41,462 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:24:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:42,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:44,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:46,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:49,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:50,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:51,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:51,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:52,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:52,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:52,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:52,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:52,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:52,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:24:52,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:24:53,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:53,275 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 14:24:53,465 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 14:24:53,526 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 14:24:53,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 14:24:54,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:55,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:55,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:56,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:24:58,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:24:59,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:00,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:01,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:02,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:04,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:06,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:06,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:25:08,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:25:08,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:09,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:09,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:11,568 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:25:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:12,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:14,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:15,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:15,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:16,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:18,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:19,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:19,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:20,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:22,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:23,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:25:23,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:25:23,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:23,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:24,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:26,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:28,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:28,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:29,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:30,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:32,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:32,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:34,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:34,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:35,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:36,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:36,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:38,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:38,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:25:38,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:25:39,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:39,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:40,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:41,640 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:25:42,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:44,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:46,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:47,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:49,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:53,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:25:53,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:25:54,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:25:54,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:54,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:55,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:25:58,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:02,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:07,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:09,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:26:09,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:26:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:09,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:11,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:11,708 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:26:15,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:19,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:21,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:26:24,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:26:24,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:25,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:29,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:30,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:31,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:33,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:37,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:39,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:39,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:26:39,713 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:26:40,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:40,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:41,773 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:26:43,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:45,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:47,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:48,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:48,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:51,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:52,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:52,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:53,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:54,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:26:55,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:26:55,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:55,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:56,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:57,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:26:59,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:26:59,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:01,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:04,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:05,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:06,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:07,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:08,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:08,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:09,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:10,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:27:10,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:27:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:10,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:11,845 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:27:11,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:11,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:13,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:15,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:15,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:16,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:16,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:17,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:19,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:19,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:20,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:20,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:21,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:22,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:22,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:23,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:24,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:25,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:27:25,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:27:27,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:28,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:31,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:32,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:33,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:34,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:34,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:35,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:36,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:36,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:38,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:39,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:41,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:27:41,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:27:41,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:41,916 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:27:45,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:45,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:51,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:27:56,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:27:56,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:27:57,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:27:57,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:01,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:03,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:03,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:05,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:09,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:28:11,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:28:12,207 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:28:12,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:12,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:14,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:14,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:16,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:20,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:22,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:24,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:24,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:26,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:26,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:27,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:28:27,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:28:28,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:28,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:32,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:32,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:34,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:38,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:40,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:42,055 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:28:42,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:28:42,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:28:42,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:43,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:43,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:46,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:46,917 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 14:28:47,099 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 14:28:47,182 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 14:28:47,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 14:28:47,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:48,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:50,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:53,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:54,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:54,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:56,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:28:57,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:28:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:57,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:58,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:28:59,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:28:59,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:00,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:02,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:04,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:05,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:05,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:06,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:08,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:09,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:09,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:09,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:09,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:09,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:09,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:10,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:12,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:29:12,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:13,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:29:13,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:29:14,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:14,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:16,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:17,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:17,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:18,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:20,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:20,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:20,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:21,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:22,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:24,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:25,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:26,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:27,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:29:28,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:29:28,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:28,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:30,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:31,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:32,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:33,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:33,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:34,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:35,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:35,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:36,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:38,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:42,216 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:29:42,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:42,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:43,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:29:43,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:29:48,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:49,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:29:54,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:58,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:29:59,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:29:59,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:30:02,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:02,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:06,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:07,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:10,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:12,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:30:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:30:14,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:30:14,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:18,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:18,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:20,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:24,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:26,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:29,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:29,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:30:30,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:30:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:30,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:31,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:32,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:32,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:33,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:35,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:39,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:41,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:42,364 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:30:43,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:43,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:43,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:43,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:43,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:45,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:30:45,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:30:45,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:46,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:46,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:49,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:51,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:53,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:54,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:55,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:57,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:30:57,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:30:59,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:00,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:00,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:01,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:31:01,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:31:01,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:03,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:03,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:04,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:05,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:06,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:07,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:08,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:09,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:09,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:11,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:12,435 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:31:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:13,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:15,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:16,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:31:16,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:31:17,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:19,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:21,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:22,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:23,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:25,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:25,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:26,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:26,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:27,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:28,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:28,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:29,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:29,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:30,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:31,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:31:31,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:31:31,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:32,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:32,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:33,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:34,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:34,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:35,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:37,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:42,509 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:31:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:31:46,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:31:47,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:50,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:50,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:53,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:31:57,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:57,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:31:58,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:32:02,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:32:02,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:05,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:05,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:06,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:08,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:09,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:12,583 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:32:12,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:14,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:14,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:16,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:17,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:32:17,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:32:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:19,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:20,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:22,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:26,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:28,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:30,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:32,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:32:32,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:32:34,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:35,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:36,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:37,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:38,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:38,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,651 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:32:42,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:42,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:43,176 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 14:32:43,380 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 14:32:43,447 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 14:32:43,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 14:32:44,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:46,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:46,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:48,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:32:48,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:32:48,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:48,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:52,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:54,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:56,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:58,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:32:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:32:58,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:01,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:02,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:03,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:33:03,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:33:03,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:03,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:04,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:05,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:06,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:07,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:07,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:08,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:10,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:11,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:12,728 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:33:12,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:13,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:14,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:16,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:16,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:18,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:33:18,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:33:18,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:20,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:21,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:21,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:22,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:23,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:24,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:26,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:27,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:27,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:28,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:29,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:29,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:30,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:31,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:31,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:32,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:33,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:33,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:33:33,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:33:35,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:37,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:42,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:33:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:47,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:49,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:33:49,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:33:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:51,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:33:52,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:52,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:33:57,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:01,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:04,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:34:04,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:34:05,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:07,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:09,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:11,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:12,877 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:34:14,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:14,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:15,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:17,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:19,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:34:19,683 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:34:20,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:20,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:21,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:23,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:23,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:27,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:29,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:30,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:30,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:31,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:34:35,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:34:35,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:37,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:37,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:39,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:39,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:39,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:40,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:41,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:42,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:34:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:43,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:45,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:47,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:49,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:49,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:50,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:34:50,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:34:51,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:53,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:55,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:34:57,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:34:57,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:00,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:00,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:01,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:03,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:04,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:05,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:35:05,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:35:05,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:07,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:08,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:08,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:09,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:11,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:13,019 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:35:13,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:13,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:14,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:15,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:16,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:17,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:19,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:20,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:35:20,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:35:21,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:21,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:22,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:23,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:25,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:27,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:27,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:27,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:28,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:29,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:30,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:30,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:31,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:31,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:32,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:33,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:35:36,137 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:35:43,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:35:43,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:47,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:49,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:49,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:51,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:35:51,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:35:54,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:55,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:55,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:35:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:58,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:59,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:59,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:59,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:59,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:59,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:35:59,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:00,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:02,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:04,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:06,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:36:06,590 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:36:08,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:10,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:11,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:12,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:13,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:36:14,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:18,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:19,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:19,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:20,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:21,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:36:21,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:36:24,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:24,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:26,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:29,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:29,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:30,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:32,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:34,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:36,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:36,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:37,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:36:37,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:36:37,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:37,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:37,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:37,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:38,417 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 14:36:38,619 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 14:36:38,688 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 14:36:38,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 14:36:39,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:39,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:40,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:42,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:43,301 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:36:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:43,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:44,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:46,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:46,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:48,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:50,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:51,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:52,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:36:52,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:36:52,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:54,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:56,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:58,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:36:59,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:36:59,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:00,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:02,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:03,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:03,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:04,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:06,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:06,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:07,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:37:07,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:37:08,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:08,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:09,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:09,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:10,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:12,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:13,368 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:37:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:14,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:14,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:15,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:16,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:18,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:19,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:19,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:20,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:21,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:21,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:22,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:22,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:37:22,978 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:37:24,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:25,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:26,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:27,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:27,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:28,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:30,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:34,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:37,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:38,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:37:38,212 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:37:40,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:43,433 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:37:46,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:46,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:50,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:54,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:37:54,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:37:54,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:54,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:37:56,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:37:58,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:02,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:04,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:05,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:05,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:06,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:08,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:09,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:38:09,653 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:38:09,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:11,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:12,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:13,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:13,506 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:38:14,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:16,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:16,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:18,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:19,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:19,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:20,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:22,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:24,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:38:24,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:38:25,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:26,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:28,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:31,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:31,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:33,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:34,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:37,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:39,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:39,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:40,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:38:40,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:38:40,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:40,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:41,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:43,575 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:38:43,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:45,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:47,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:48,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:48,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:49,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:50,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:51,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:52,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:52,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:53,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:38:55,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:38:55,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:56,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:38:57,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:38:59,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:00,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:00,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:01,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:03,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:04,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:05,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:06,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:06,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:07,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:09,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:10,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:39:10,735 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:39:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:11,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:12,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,643 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:39:13,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:13,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:15,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:17,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:18,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:19,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:21,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:21,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:23,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:23,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:24,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:24,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:25,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:26,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:39:26,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:39:35,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:35,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:38,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:38,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:39:41,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:39:41,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:43,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:43,711 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:39:44,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:44,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:45,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:48,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:48,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:49,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:51,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:55,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:39:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:39:56,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:39:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:39:59,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:01,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:05,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:07,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:07,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:09,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:11,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:11,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:40:12,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:40:13,783 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:40:13,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:14,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:17,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:18,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:18,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:19,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:21,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:23,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:23,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:25,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:27,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:40:27,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:40:27,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:28,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 14:40:29,045 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 14:40:29,215 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 14:40:29,272 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 14:40:30,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 14:40:30,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:32,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:34,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:36,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:38,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:40,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:42,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:42,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:40:42,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:40:43,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:40:44,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:46,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:48,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:50,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:52,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:54,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:57,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:40:57,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:40:57,885 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:40:59,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:01,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:03,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:05,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:07,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:09,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:11,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:13,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:13,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:41:13,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:41:13,939 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:41:15,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:17,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:19,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:21,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:23,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:25,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:27,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:28,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:41:28,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:41:29,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:31,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:33,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:35,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:37,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:39,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:41,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:43,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:43,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:41:43,720 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:41:44,015 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:41:45,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:47,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:49,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:51,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:53,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:55,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:57,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:41:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:41:59,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:41:59,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:01,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:04,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:06,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:08,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:10,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:12,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:14,094 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:42:14,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:42:14,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:42:16,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:18,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:20,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:22,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:24,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:26,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:28,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:29,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:42:29,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:42:30,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:32,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:34,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:36,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:38,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:40,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:42,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:44,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:44,168 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:42:44,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:42:44,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:42:46,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:48,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:50,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:52,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:54,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:56,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:42:58,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:00,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:43:00,079 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:43:00,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:02,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:04,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:06,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:08,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:10,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:12,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:14,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:14,247 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:43:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:43:15,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:43:16,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:18,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:20,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:22,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:24,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:26,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:28,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:30,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:30,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:43:30,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:43:32,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:34,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:36,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:39,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:41,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:43,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:44,305 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:43:45,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:45,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:43:45,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:43:47,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:49,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:51,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:53,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:55,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:57,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:43:59,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:01,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:44:01,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:44:01,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:03,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:05,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:07,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:09,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:11,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:13,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:14,375 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:44:15,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:16,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:44:16,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:44:17,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:19,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:21,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:23,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:25,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:27,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:29,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:31,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:44:31,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:44:33,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:35,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:37,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:39,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:41,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:43,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:44,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:44:45,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:47,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:44:47,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:44:47,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:49,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:51,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:53,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:55,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:57,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:44:59,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:01,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:02,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:45:02,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:45:03,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:05,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:07,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:09,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:11,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:13,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:14,524 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:45:15,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:17,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:17,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:45:17,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:45:19,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:21,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:23,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:26,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:28,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:30,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:32,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:32,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:45:32,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:45:34,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:36,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:38,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:40,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:42,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:44,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:44,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:45:46,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:48,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:48,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:45:48,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:45:50,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:52,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:54,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:56,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:45:58,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:00,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:02,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:03,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:46:03,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:46:04,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:06,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:08,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:10,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:12,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:14,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:14,673 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:46:16,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:18,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:18,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:46:18,977 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:46:20,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:22,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:24,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:26,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:28,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:30,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:32,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:34,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:34,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:46:34,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:46:36,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:38,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:40,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:42,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:44,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:44,750 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:46:46,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:48,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:49,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:46:49,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:46:50,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:52,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:54,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:56,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:46:58,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:00,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:02,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:04,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:04,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:47:04,981 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:47:06,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:08,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:11,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:13,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:14,823 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:47:15,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:17,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:19,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:20,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:47:20,343 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:47:21,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:23,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:25,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:27,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:29,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:31,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:33,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:35,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:35,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:47:35,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:47:37,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:39,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:41,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:43,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:44,889 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:47:45,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:47,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:49,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:50,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:47:50,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:47:51,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:53,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:55,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:57,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:47:59,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:01,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:03,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:05,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:48:06,122 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:48:07,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:09,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:11,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:13,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:14,958 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:48:15,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:17,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:19,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:21,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:21,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:48:21,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:48:23,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:25,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:27,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:29,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:31,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:33,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:35,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:36,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:48:36,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:48:37,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:39,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:41,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:43,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:45,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:48:45,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:47,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:50,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:48:51,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:48:52,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:54,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:56,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:48:58,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:00,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:02,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:04,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:06,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:07,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:49:07,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:49:08,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:10,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:12,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:14,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:15,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:49:16,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:18,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:20,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:22,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:22,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:49:22,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:49:24,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:26,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:28,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:30,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:32,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:34,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:36,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:37,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:49:37,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:49:38,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:40,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:42,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:44,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:45,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:49:46,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:48,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:50,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:52,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:49:53,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:49:54,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:56,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:49:58,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:00,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:02,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:04,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:06,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:08,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:08,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:50:08,367 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:50:10,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:13,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:15,228 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:50:15,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:17,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:19,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:21,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:23,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:23,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:50:23,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:50:25,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:27,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:29,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:31,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:33,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:35,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:37,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:50:38,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:50:39,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:41,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:43,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:45,302 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:50:45,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:47,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:49,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:51,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:53,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:50:54,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:50:55,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:57,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:50:59,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:01,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:03,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:05,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:07,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:09,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:09,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:51:09,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:51:11,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:13,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:15,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:51:15,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:17,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:19,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:21,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:23,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:25,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:51:25,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:51:25,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:27,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:29,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:31,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:33,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:35,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:38,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:40,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:51:40,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:51:40,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:42,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:44,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:45,446 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:51:46,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:48,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:50,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:52,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:54,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:55,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:51:55,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:51:56,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:51:58,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:00,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:02,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:04,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:06,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:08,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:10,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:11,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:52:11,109 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:52:12,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:14,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:15,519 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:52:16,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:18,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:20,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:22,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:24,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:26,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:52:26,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:52:26,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:28,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:30,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:32,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:34,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:36,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:38,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:40,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:52:41,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:52:42,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:44,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:45,594 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:52:46,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:48,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:50,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:52,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:54,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:56,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:52:56,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:52:56,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:52:58,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:00,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:02,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:04,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:06,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:09,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:11,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:12,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:53:12,189 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:53:13,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:15,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:15,670 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:53:17,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:19,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:21,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:23,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:25,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:27,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:27,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:53:27,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:53:29,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:31,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:33,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:35,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:37,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:39,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:41,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:42,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:53:42,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:53:43,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:45,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:45,744 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:53:47,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:49,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:51,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:53,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:55,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:57,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:53:58,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:53:58,012 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:53:59,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:01,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:03,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:05,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:07,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:09,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:11,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:13,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:54:13,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:54:13,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:15,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:15,818 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:54:17,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:19,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:21,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:24,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:26,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:28,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:28,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:54:28,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:54:30,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:32,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:34,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:36,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:38,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:40,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:42,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:54:44,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:54:44,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:45,894 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:54:46,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:48,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:50,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:52,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:54,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:56,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:58,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:54:59,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:54:59,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:55:00,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:02,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:04,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:06,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:08,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:10,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:12,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:14,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:14,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:55:14,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:55:15,969 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:55:16,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:18,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:20,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:22,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:24,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:26,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:28,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:55:30,149 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:55:30,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:32,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:34,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:36,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:38,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:40,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:42,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:44,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:45,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:55:45,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:55:46,048 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:55:46,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:48,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:50,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:52,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:54,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:56,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:55:58,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:00,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:00,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:56:00,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:56:02,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:04,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:06,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:08,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:10,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:12,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:14,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:16,117 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:56:16,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:56:16,160 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:56:16,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:18,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:21,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:23,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:25,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:27,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:29,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:56:31,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:56:31,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:33,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:35,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:37,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:39,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:41,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:43,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:45,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:46,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:56:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:56:46,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:56:47,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:49,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:51,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:53,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:55,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:57,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:56:59,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:01,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:02,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:57:02,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:57:03,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:05,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:07,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:09,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:11,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:13,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:15,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:16,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:57:17,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:57:17,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:57:17,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:19,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:21,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:23,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:25,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:27,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:29,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:31,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:32,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:57:32,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:57:33,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:35,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:37,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:39,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:41,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:43,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:45,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:46,337 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:57:47,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:48,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:57:48,049 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:57:49,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:51,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:53,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:55,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:57:58,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:00,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:02,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:03,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:58:03,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:58:04,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:06,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:08,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:10,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:12,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:14,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:16,410 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:58:16,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:18,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:18,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:58:18,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:58:20,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:22,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:24,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:26,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:28,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:30,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:32,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:34,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:58:34,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:58:34,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:36,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:38,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:40,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:42,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:44,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:46,486 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:58:46,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:48,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:49,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:58:49,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:58:50,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:52,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:54,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:56,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:58:58,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:00,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:02,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:04,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:04,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:59:04,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:59:06,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:08,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:10,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:12,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:14,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:16,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:59:16,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:18,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:20,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:59:20,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:59:20,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:22,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:24,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:26,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:28,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:31,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:33,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:35,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:59:35,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:59:35,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:37,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:39,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:41,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:43,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:45,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:46,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 14:59:47,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:49,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:50,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 14:59:50,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 14:59:51,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:53,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:55,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:57,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 14:59:59,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:01,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:03,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:05,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:00:06,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:00:07,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:09,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:11,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:13,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:15,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:16,716 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:00:17,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:19,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:21,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:21,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:00:21,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:00:23,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:25,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:27,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:29,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:31,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:33,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:35,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:37,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:00:37,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:00:37,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:39,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:41,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:43,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:45,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:46,785 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:00:47,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:49,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:51,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:52,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:00:52,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:00:53,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:55,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:57,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:00:59,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:01,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:03,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:05,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:07,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:07,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:01:07,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:01:09,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:11,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:13,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:15,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:16,856 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:01:17,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:19,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:21,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:23,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:01:23,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:01:23,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:25,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:27,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:29,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:31,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:33,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:35,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:38,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:01:38,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:01:40,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:42,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:44,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:46,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:46,933 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:01:48,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:50,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:52,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:54,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:01:54,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:01:54,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:56,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:01:58,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:00,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:02,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:04,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:06,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:08,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:09,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:02:09,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:02:10,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:12,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:14,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:16,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:17,009 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:02:18,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:20,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:22,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:24,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:02:24,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:02:24,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:26,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:28,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:30,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:32,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:34,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:36,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:38,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:39,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:02:39,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:02:40,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:42,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:44,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:46,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:47,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:02:48,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:50,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:52,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:02:55,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:02:55,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:57,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:02:59,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:01,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:03,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:05,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:07,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:09,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:10,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:03:10,462 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:03:11,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:13,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:15,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:17,158 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:03:17,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:19,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:21,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:23,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:25,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:03:25,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:03:27,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:29,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:31,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:33,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:35,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:37,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:39,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:41,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:03:41,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:03:41,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:43,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:45,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:47,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:03:47,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:49,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:51,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:53,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:55,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:56,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:03:56,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:03:57,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:03:59,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:01,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:03,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:06,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:08,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:10,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:11,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:04:11,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:04:12,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:14,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:16,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:17,293 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:04:18,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:20,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:22,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:24,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:26,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:26,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:04:26,925 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:04:28,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:30,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:32,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:34,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:36,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:38,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:40,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:42,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:04:42,163 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:04:42,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:44,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:46,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:47,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:04:48,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:50,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:52,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:54,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:56,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:04:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:04:57,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:04:58,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:00,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:02,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:04,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:06,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:08,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:10,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:12,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:12,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:05:12,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:05:14,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:16,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:17,448 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:05:19,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:21,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:23,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:25,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:27,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:28,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:05:28,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:05:29,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:31,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:33,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:35,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:37,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:39,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:41,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:43,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:05:43,363 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:05:43,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:45,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:47,525 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:05:47,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:49,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:51,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:53,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:55,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:57,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:05:58,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:05:58,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:05:59,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:01,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:03,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:05,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:07,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:09,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:11,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:13,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:06:13,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:06:15,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:17,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:06:17,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:19,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:21,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:23,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:25,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:27,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:29,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:06:30,580 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:06:31,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:34,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:36,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:38,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:40,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:42,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:44,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:45,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:06:45,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:06:46,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:47,682 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:06:48,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:50,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:52,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:54,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:56,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:06:58,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:00,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:01,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:07:01,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:07:02,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:04,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:06,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:08,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:10,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:12,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:14,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:16,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:07:16,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:07:16,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:17,755 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:07:18,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:20,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:22,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:24,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:26,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:28,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:30,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:31,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:07:31,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:07:32,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:34,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:37,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:39,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:41,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:43,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:45,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:47,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:07:47,137 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:07:47,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:47,830 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:07:49,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:51,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:53,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:55,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:57,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:07:59,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:01,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:08:02,409 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:08:03,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:05,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:07,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:09,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:11,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:13,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:15,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:17,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:08:17,699 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:08:17,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:17,925 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:08:19,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:21,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:23,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:25,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:27,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:29,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:31,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:32,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:08:32,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:08:33,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:35,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:37,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:39,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:41,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:43,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:45,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:47,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:47,979 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:08:48,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:08:48,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:08:49,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:51,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:53,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:55,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:57,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:08:59,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:01,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:03,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:09:03,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:09:03,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:05,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:07,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:09,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:11,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:13,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:15,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:17,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:18,057 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:09:19,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:09:19,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:09:20,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:22,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:24,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:26,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:28,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:30,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:32,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:34,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:35,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:09:35,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:09:36,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:38,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:40,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:42,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:44,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:46,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:48,133 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:09:48,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:09:50,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:09:50,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:52,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:54,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:56,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:09:58,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:00,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:02,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:04,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:10:05,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:10:06,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:08,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:10,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:12,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:14,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:16,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:18,210 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:10:18,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:20,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:21,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:10:21,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:10:22,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:24,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:26,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:28,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:30,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:32,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:34,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:36,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:10:36,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:10:36,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:38,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:40,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:42,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:44,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:46,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:48,286 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:10:48,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:50,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:51,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:10:51,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:10:52,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:55,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:57,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:10:59,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:01,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:03,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:05,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:07,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:11:07,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:11:07,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:09,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:11,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:13,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:15,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:17,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:18,358 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:11:19,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:21,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:11:22,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:11:23,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:25,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:27,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:29,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:31,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:33,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:35,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:37,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:11:37,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:11:37,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:39,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:41,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:43,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:45,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:47,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:48,435 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:11:49,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:51,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:52,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:11:52,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:11:53,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:55,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:57,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:11:59,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:01,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:03,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:05,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:07,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:12:08,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:12:09,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:11,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:13,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:15,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:17,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:18,507 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:12:19,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:21,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:23,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:12:23,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:12:23,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:25,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:27,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:29,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:31,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:33,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:35,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:37,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:38,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:12:38,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:12:39,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:42,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:44,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:46,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:48,581 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:12:48,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:50,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:52,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:54,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:12:54,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:12:54,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:56,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:12:58,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:00,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:02,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:04,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:06,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:08,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:09,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:13:09,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:13:10,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:12,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:14,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:16,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:18,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:13:18,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:20,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:22,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:24,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:13:24,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:13:24,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:26,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:28,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:30,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:32,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:34,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:36,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:38,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:39,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:13:39,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:13:40,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:42,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:44,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:46,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:48,728 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:13:48,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:50,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:52,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:54,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:55,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:13:55,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:13:56,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:13:58,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:00,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:02,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:04,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:06,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:08,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:10,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:14:10,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:14:11,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:13,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:15,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:17,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:18,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:14:19,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:21,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:23,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:25,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:14:25,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:14:25,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:27,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:29,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:31,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:33,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:35,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:37,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:39,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:41,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:14:41,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:14:41,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:43,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:45,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:47,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:48,881 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:14:49,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:51,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:53,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:55,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:56,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:14:56,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:14:57,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:14:59,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:01,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:03,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:05,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:08,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:10,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:15:11,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:15:12,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:14,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:16,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:18,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:18,957 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:15:20,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:22,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:24,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:26,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:15:26,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:15:28,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:30,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:32,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:34,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:36,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:38,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:40,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:42,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:42,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:15:42,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:15:44,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:46,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:48,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:49,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:15:50,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:52,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:54,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:56,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:15:57,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:15:57,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:15:58,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:00,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:02,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:05,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:07,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:09,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:11,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:16:12,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:16:13,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:15,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:17,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:19,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:19,113 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:16:21,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:23,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:25,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:27,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:28,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:16:28,336 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:16:29,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:31,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:33,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:35,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:37,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:39,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:41,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:43,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:43,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:16:43,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:16:45,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:47,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:49,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:49,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:16:51,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:53,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:55,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:57,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:59,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:16:59,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:16:59,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:17:01,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:03,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:05,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:07,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:09,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:11,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:13,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:14,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:17:14,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:17:15,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:17,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:19,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:19,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:17:21,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:23,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:26,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:28,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:29,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:17:29,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:17:30,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:32,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:34,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:36,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:38,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:40,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:42,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:44,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:45,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:17:45,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:17:46,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:48,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:49,335 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:17:50,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:52,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:54,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:56,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:17:58,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:00,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:18:00,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:18:02,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:04,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:06,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:08,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:10,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:12,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:14,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:15,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:18:15,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:18:16,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:18,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:19,413 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:18:20,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:22,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:24,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:26,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:28,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:30,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:31,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:18:31,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:18:32,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:34,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:36,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:38,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:40,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:42,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:44,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:46,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:46,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:18:46,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:18:48,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:49,485 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:18:50,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:52,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:54,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:56,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:18:58,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:00,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:01,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:19:01,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:19:03,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:05,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:07,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:09,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:11,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:13,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:15,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:17,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:17,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:19:17,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:19:19,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:19,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:19:21,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:23,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:25,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:27,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:29,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:31,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:32,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:19:32,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:19:33,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:35,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:37,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:39,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:41,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:43,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:45,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:47,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:48,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:19:48,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:19:49,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:49,636 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:19:51,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:53,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:55,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:57,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:19:59,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:01,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:03,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:03,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:20:03,900 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:20:05,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:07,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:09,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:11,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:13,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:15,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:17,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:19,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:20:19,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:20:19,712 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:20:21,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:23,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:25,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:27,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:29,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:31,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:33,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:20:34,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:20:35,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:37,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:39,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:41,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:43,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:45,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:47,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:49,788 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:20:50,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:20:50,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:20:52,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:54,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:56,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:20:58,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:00,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:02,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:04,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:06,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:21:06,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:21:06,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:08,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:10,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:12,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:14,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:16,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:18,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:19,863 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:21:20,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:21,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:21:21,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:21:22,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:24,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:26,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:28,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:30,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:32,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:34,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:36,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:36,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:21:36,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:21:38,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:40,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:42,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:44,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:46,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:48,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:49,940 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:21:50,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:52,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:52,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:21:52,497 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:21:54,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:56,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:21:59,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:01,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:03,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:05,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:07,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:22:07,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:22:09,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:11,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:13,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:15,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:17,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:19,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:22:20,016 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:22:23,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:22:23,117 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:22:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:22:39,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:22:50,091 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:22:54,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:22:54,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:23:10,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:23:10,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:23:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:23:17,166 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 15:23:17,215 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 15:23:17,290 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 15:23:18,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 15:23:19,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:23:20,199 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:23:23,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:23:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:23:25,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:23:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:23:41,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:23:41,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:23:50,283 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:23:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:23:56,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:24:11,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:24:11,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:24:20,367 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:24:26,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:24:26,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:24:42,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:24:42,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:24:50,446 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:24:57,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:24:57,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:25:12,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:25:12,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:25:20,521 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:25:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:25:32,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:25:48,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:25:48,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:25:50,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:26:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:26:04,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:26:19,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:26:19,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:26:20,675 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:26:34,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:26:34,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:26:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:26:50,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:26:50,753 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:27:05,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:27:05,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:27:20,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:27:20,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:27:20,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:27:35,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:27:35,932 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:27:37,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:40,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:40,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:41,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:43,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:43,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:45,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:47,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:47,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:49,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:49,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:49,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:50,904 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:27:51,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:27:51,206 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:27:51,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:53,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:53,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:53,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:55,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:57,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:58,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:58,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:27:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:27:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:01,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:02,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:02,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:03,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:05,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:06,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:28:06,602 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:28:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:07,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:09,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:11,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:11,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:11,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:12,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:13,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:14,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:15,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:16,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:17,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:17,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:19,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:20,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:20,977 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:28:21,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:28:21,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:28:22,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:22,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:24,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:24,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:26,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:28,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:35,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:37,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:28:37,251 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:28:38,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:38,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:50,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:51,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:28:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:28:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:28:52,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:28:54,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:28:58,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:02,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:04,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:29:08,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:29:08,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:10,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:10,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:11,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:14,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:16,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:17,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:19,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:20,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:21,121 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:29:22,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:23,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:29:23,830 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:29:24,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:25,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:25,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:28,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:29,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:30,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:32,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:34,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:36,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:38,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:40,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:29:40,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:29:40,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:40,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:41,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:42,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:44,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:46,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:46,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:48,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:50,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:50,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:51,189 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:29:52,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:54,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:55,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:55,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:29:56,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:29:56,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:56,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:58,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:29:59,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:29:59,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:00,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:00,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:03,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:04,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:04,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:05,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:06,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:07,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:08,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:08,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:09,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:10,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:11,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:30:11,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:30:11,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:11,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:12,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:13,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:14,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:14,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:15,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:17,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:18,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:19,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:21,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:30:21,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:27,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:30:27,606 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:30:31,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:32,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:37,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:38,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:41,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:44,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:30:44,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:30:44,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:44,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:45,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:46,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:49,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:50,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:51,331 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:30:52,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:52,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:53,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:55,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:30:56,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:56,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:30:59,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:00,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:31:00,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:31:03,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:04,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:04,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:05,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:09,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:15,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:17,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:17,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:17,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:31:17,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:31:19,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:19,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:19,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:21,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:31:23,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:23,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:23,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:25,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:25,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:26,774 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 15:31:26,979 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 15:31:27,048 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 15:31:27,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 15:31:27,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:27,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:28,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:29,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:33,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:31:33,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:31:33,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:34,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:35,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:37,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:40,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:40,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:41,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:42,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:43,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:44,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:46,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:48,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:48,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:49,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:49,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:31:49,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:31:50,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:51,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:51,504 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:31:52,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:54,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:56,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:31:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:56,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:31:58,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:00,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:02,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:03,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:03,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:04,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:05,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:32:05,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:32:06,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:06,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:07,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:07,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:08,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:08,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:09,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:10,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:11,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:12,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:14,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:14,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:16,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:16,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:16,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:17,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:18,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:20,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:32:20,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:32:21,710 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:32:21,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:21,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:28,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:30,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:32,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:36,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:36,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:32:36,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:32:40,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:41,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:44,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:45,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:48,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:49,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:49,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,643 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:32:51,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:51,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:52,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:32:52,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:32:52,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:54,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:56,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:58,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:32:59,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:32:59,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:01,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:02,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:04,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:07,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:33:07,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:33:08,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:09,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:10,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:10,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:10,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:10,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:10,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:12,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:13,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:13,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:14,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:16,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:18,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:18,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:20,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:21,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:21,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:21,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:21,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:21,717 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:33:22,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:23,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:33:23,629 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:33:24,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:25,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:26,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:27,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:28,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:29,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:29,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:30,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:32,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:34,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:36,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:38,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:33:38,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:33:39,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:40,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:41,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:43,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:45,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:47,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:48,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:48,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:49,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:49,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:51,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:51,787 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:33:53,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:54,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:33:54,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:33:55,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:56,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:56,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:57,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:59,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:33:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:33:59,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:01,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:02,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:03,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:05,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:05,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:06,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:07,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:07,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:08,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:09,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:09,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:34:09,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:34:10,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:11,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:11,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:13,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:14,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:15,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:17,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:21,855 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:34:24,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:24,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:34:26,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:34:27,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:27,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:31,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:33,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:33,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:35,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:39,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:40,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:41,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:34:41,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:34:43,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:47,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:49,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:49,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:51,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:51,930 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:34:52,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:52,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:53,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:53,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:53,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:53,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:53,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:54,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:57,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:34:57,238 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:34:57,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:34:59,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:34:59,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:01,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:02,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:02,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:03,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:05,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:07,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:08,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:09,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:12,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:35:12,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:35:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:13,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:15,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:16,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:16,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:17,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:20,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:21,800 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 15:35:22,002 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 15:35:22,067 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 15:35:22,071 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:35:22,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:22,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 15:35:23,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:23,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:23,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:25,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:27,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:28,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:35:28,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:35:28,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:28,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:29,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:31,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:32,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:33,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:33,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:34,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:36,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:37,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:37,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:38,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:40,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:42,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:42,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:43,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:35:43,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:35:43,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:43,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:44,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:46,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:48,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:48,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:50,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:50,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:51,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:52,143 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:35:52,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:54,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:54,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:56,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:57,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:57,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:35:58,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:35:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:35:59,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:36:00,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:01,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:01,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:02,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:04,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:06,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:06,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:07,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:08,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:10,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:11,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:12,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:12,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:36:14,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:36:14,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:22,216 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:36:24,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:28,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:29,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:36:29,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:36:32,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:36,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:37,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:37,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:40,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:44,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:45,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:36:45,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:36:48,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:50,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:50,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:52,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:36:54,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:56,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:36:56,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:36:58,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:00,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:00,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:37:00,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:37:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:01,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:02,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:05,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:06,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:07,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:07,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:08,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:10,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:11,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:11,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:14,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:14,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:16,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:37:16,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:37:16,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:17,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:18,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:19,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:22,364 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:37:22,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:24,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:26,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:27,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:27,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:28,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:30,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:30,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:37:31,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:37:31,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:31,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:32,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:34,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:34,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:34,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:36,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:38,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:38,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:38,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:40,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:41,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:41,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:42,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:44,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:46,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:37:47,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:37:48,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:49,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:49,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:51,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:52,438 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:37:52,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:52,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:53,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:55,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:57,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:37:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:37:59,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:01,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:02,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:38:02,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:38:03,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:03,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:05,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:07,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:08,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:09,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:11,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:12,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:17,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:17,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:38:17,795 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:38:20,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:20,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:38:22,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:23,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:26,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:27,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:28,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:31,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:38:33,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:38:35,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:37,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:38,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:39,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:41,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:45,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:48,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:38:48,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:38:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:49,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:51,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:52,584 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:38:55,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:57,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:38:57,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:57,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:58,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:38:59,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:00,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:00,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:01,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:03,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:39:04,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:39:05,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:06,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:07,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:10,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:11,738 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 15:39:11,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:11,951 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 15:39:12,021 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 15:39:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:12,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 15:39:12,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:13,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:14,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:14,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:15,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:17,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:18,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:18,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:19,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:39:19,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:39:19,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:21,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:22,667 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:39:23,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:25,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:25,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:27,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:28,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:29,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:30,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:31,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:32,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:32,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:33,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:35,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:39:35,035 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:39:35,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:37,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:37,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:38,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:39,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:41,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:41,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:43,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:45,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:46,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:47,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:48,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:48,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:49,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:39:50,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:39:50,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:50,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:51,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:52,741 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:39:53,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:54,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:55,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:57,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:39:58,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:58,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:39:59,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:00,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:01,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:01,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:02,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:04,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:05,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:40:05,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:40:06,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:07,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:07,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:14,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:15,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:15,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:19,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:20,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:40:20,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:40:22,817 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:40:24,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:28,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:28,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:28,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:32,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:32,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:40:36,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:40:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:36,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:40,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:42,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:42,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:43,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:44,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:46,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:47,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:47,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:48,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:52,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:40:52,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:52,892 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:40:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:40:53,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:40:55,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:40:59,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:00,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:00,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:01,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:03,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:04,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:04,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:06,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:07,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:08,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:41:08,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:41:08,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:08,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:09,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:10,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:11,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:13,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:14,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:15,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:17,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:17,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:18,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:19,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:21,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:22,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:41:23,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:23,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:41:23,884 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:41:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:25,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:27,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:29,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:30,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:31,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:33,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:33,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:35,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:37,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:39,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:41:39,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:41:39,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:39,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:40,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:41,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:43,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:45,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:46,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:47,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:49,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:50,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:50,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:51,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:53,041 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:41:53,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:53,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:54,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:41:54,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:41:55,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:55,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:57,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:58,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:58,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:41:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:41:59,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:00,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:01,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:42:09,977 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:42:11,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:17,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:17,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:20,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:20,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:21,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:23,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:42:25,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:42:25,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:42:25,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:25,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:27,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:27,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:29,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:30,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:30,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:31,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:32,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:34,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:35,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:39,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:39,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:40,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:42:40,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:42:41,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:44,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:46,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:48,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:50,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:52,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:53,192 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:42:54,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:56,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:42:56,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:42:56,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:42:59,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:42:59,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:00,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:02,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:04,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:05,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:05,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,072 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 15:43:06,277 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 15:43:06,350 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 15:43:06,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:06,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 15:43:07,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:08,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:09,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:09,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:10,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:11,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:43:11,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:43:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:11,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:12,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:14,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:14,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:16,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:17,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:18,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:19,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:20,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:22,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:22,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:23,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:43:24,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:24,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:26,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:43:26,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:43:26,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:27,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:27,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:28,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:30,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:30,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:31,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:32,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:34,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:36,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:38,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:40,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:42,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:43:42,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:43:42,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:44,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:45,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:45,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:46,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:48,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:48,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:50,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:52,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:53,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:43:53,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:53,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:54,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:54,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:43:56,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:43:58,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:43:58,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:43:58,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:01,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:06,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:07,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:12,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:13,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:44:13,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:44:16,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:20,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:23,430 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:44:24,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:24,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:26,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:26,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:28,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:29,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:44:29,057 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:44:31,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:32,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:33,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:33,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:34,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:37,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:38,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:40,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:42,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:42,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:43,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:44,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:44:44,479 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:44:44,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:46,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:47,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:49,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:50,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:52,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:53,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,496 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:44:53,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:53,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:44:54,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:58,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:44:59,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:44:59,748 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:45:00,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:00,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:00,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:01,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:02,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:04,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:04,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:06,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:07,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:10,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:12,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:13,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:13,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:14,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:15,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:45:15,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:45:15,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:16,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:18,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:19,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:19,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:20,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:22,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:23,566 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:45:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:24,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:25,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:26,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:27,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:29,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:30,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:45:30,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:45:31,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:32,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:33,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:34,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:34,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:35,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:36,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:37,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:39,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:41,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:42,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:42,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:43,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:44,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:44,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:45:45,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:45:45,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:47,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:49,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:49,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:50,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:45:51,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:45:53,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:45:53,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:46:01,107 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:46:01,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:01,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:05,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:07,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:07,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:09,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:13,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:13,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:16,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:46:16,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:46:17,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:21,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:23,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:23,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:46:25,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:25,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:27,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:28,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:28,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:31,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:46:32,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:46:33,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:36,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:37,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:39,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:39,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:43,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:44,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:45,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:46:47,335 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:46:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:48,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:49,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:50,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:50,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:51,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:53,783 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:46:53,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:54,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:54,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:54,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 15:46:55,717 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 15:46:55,919 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 15:46:55,978 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 15:46:55,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 15:46:55,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:57,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:46:59,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:01,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:03,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:47:03,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:47:03,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:06,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:08,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:10,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:12,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:14,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:16,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:18,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:47:18,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:47:18,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:20,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:22,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:24,050 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:47:24,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:26,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:28,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:30,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:32,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:34,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:47:34,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:47:34,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:36,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:38,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:40,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:42,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:44,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:47,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:49,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:50,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:47:50,344 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:47:51,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:53,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:54,130 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:47:55,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:57,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:47:59,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:01,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:03,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:05,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:48:05,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:48:07,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:09,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:11,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:14,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:16,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:18,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:20,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:48:20,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:48:22,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:24,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:24,204 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:48:26,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:28,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:30,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:32,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:34,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:36,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:36,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:48:36,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:48:38,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:40,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:42,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:44,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:46,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:48,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:50,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:48:51,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:48:52,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:54,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:54,278 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:48:56,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:48:58,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:00,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:02,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:04,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:06,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:08,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:49:08,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:49:10,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:12,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:14,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:16,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:18,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:20,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:22,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:23,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:49:23,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:49:24,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:24,355 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:49:26,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:28,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:30,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:32,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:34,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:36,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:39,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:39,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:49:39,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:49:41,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:43,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:45,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:47,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:49,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:51,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:53,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:54,434 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:49:54,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:49:54,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:49:55,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:57,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:49:59,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:01,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:03,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:05,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:07,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:09,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:10,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:50:10,268 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:50:11,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:13,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:15,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:17,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:19,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:21,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:23,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:24,510 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:50:25,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:25,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:50:25,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:50:27,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:29,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:31,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:33,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:35,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:37,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:39,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:41,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:42,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:50:42,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:50:43,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:45,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:47,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:50,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:52,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:54,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:54,587 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:50:56,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:50:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:50:58,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:50:58,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:00,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:02,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:04,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:06,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:08,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:10,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:12,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:51:13,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:51:14,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:16,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:18,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:20,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:22,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:24,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:24,662 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:51:26,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:28,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:29,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:51:29,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:51:30,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:32,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:34,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:36,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:38,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:40,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:42,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:44,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:44,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:51:44,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:51:46,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:48,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:50,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:52,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:54,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:54,738 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:51:56,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:51:58,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:00,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:00,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:52:00,137 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:52:02,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:04,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:06,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:08,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:10,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:12,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:14,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:15,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:52:15,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:52:16,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:18,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:20,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:22,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:24,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:24,819 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:52:27,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:29,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:31,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:31,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:52:31,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:52:33,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:35,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:37,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:39,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:41,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:43,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:45,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:46,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:52:46,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:52:47,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:49,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:51,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:53,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:54,897 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:52:55,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:57,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:52:59,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:01,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:02,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:53:02,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:53:03,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:05,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:07,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:09,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:11,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:13,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:15,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:17,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:17,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:53:17,606 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:53:19,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:21,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:23,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:24,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:53:25,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:27,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:29,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:31,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:53:32,884 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:53:33,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:35,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:37,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:39,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:41,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:43,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:45,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:47,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:53:48,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:53:49,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:51,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:53,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:55,049 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:53:55,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:57,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:53:59,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:01,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:03,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:03,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:54:03,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:54:05,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:07,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:09,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:11,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:13,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:16,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:18,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:18,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:54:18,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:54:20,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:22,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:24,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:25,125 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:54:26,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:28,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:30,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:32,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:34,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:34,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:54:34,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:54:36,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:38,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:40,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:42,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:44,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:46,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:48,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:49,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:54:49,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:54:50,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:52,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:54,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:55,201 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:54:56,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:54:58,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:00,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:02,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:04,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:55:04,806 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:55:06,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:08,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:10,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:12,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:14,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:16,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:18,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:20,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:55:20,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:55:20,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:22,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:24,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:25,276 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:55:27,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:29,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:31,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:33,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:35,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:35,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:55:35,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:55:37,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:39,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:41,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:43,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:45,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:47,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:49,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:50,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:55:50,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:55:51,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:53,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:55,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:55,349 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:55:57,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:55:59,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:01,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:03,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:05,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:06,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:56:06,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:56:07,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:09,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:11,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:13,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:15,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:17,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:19,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:21,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:21,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:56:21,593 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:56:23,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:25,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:25,427 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:56:27,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:29,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:31,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:33,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:35,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:36,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:56:36,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:56:37,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:39,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:41,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:43,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:45,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:47,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:49,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:51,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:56:52,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:56:53,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:55,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:55,503 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:56:57,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:56:59,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:01,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:04,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:06,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:07,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:57:07,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:57:08,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:10,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:12,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:14,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:16,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:18,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:20,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:22,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:57:22,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:57:24,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:25,578 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:57:26,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:28,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:30,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:32,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:34,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:36,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:38,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:57:38,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:57:38,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:40,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:42,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:44,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:46,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:48,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:50,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:52,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:53,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:57:53,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:57:54,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:55,654 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:57:56,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:57:58,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:00,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:02,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:04,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:06,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:08,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:09,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:58:09,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:58:10,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:12,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:14,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:16,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:18,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:20,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:23,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:24,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:58:24,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:58:25,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:25,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:58:27,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:29,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:31,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:33,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:35,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:37,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:39,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:39,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:58:39,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:58:41,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:43,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:45,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:47,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:49,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:51,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:53,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:55,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:55,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:58:55,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:58:55,804 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:58:57,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:58:59,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:01,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:03,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:05,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:07,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:09,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:10,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:59:10,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:59:11,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:13,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:15,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:17,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:19,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:21,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:23,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:25,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:25,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:59:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:59:26,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:59:27,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:29,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:31,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:33,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:35,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:37,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:39,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:41,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:41,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:59:41,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:59:43,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:45,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:47,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:49,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:51,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:53,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:55,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:55,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 15:59:56,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 15:59:56,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 15:59:57,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 15:59:59,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:01,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:03,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:05,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:07,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:10,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:12,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:00:12,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:00:12,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:14,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:16,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:18,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:20,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:22,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:24,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:26,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:00:26,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:00:27,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:00:28,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:30,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:32,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:34,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:36,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:38,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:40,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:42,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:44,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:45,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:00:45,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:00:46,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:48,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:50,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:52,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:54,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:56,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:00:56,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:00:58,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:00,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:00,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:01:00,887 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:01:02,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:04,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:06,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:08,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:10,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:12,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:14,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:16,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:17,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:01:17,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:01:18,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:20,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:22,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:24,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:26,181 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:01:27,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:29,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:31,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:33,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:33,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:01:33,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:01:35,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:37,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:39,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:41,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:43,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:45,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:47,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:48,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:01:48,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:01:49,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:51,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:53,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:55,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:56,255 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:01:57,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:01:59,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:01,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:03,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:04,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:02:04,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:02:05,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:07,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:09,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:11,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:13,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:15,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:17,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:19,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:19,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:02:19,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:02:21,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:23,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:25,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:26,332 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:02:27,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:29,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:31,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:33,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:02:35,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:02:35,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:37,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:39,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:41,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:43,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:45,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:47,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:49,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:50,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:02:50,431 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:02:51,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:53,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:55,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:56,408 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:02:57,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:02:59,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:01,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:03,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:05,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:05,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:03:05,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:03:07,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:09,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:11,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:13,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:15,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:17,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:19,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:03:21,255 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:03:21,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:23,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:25,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:26,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:03:27,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:29,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:32,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:34,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:36,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:37,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:03:37,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:03:38,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:40,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:42,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:44,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:46,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:48,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:50,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:03:52,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:03:52,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:54,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:56,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:03:56,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:03:58,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:00,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:02,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:04,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:06,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:04:07,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:04:08,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:10,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:12,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:14,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:16,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:18,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:20,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:22,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:23,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:04:23,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:04:24,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:26,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:26,637 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:04:28,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:30,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:32,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:34,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:36,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:38,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:04:38,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:04:40,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:42,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:44,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:46,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:48,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:50,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:52,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:04:53,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:04:54,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:56,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:04:56,693 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:04:58,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:00,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:02,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:05,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:07,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:09,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:05:09,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:05:09,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:11,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:13,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:15,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:17,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:19,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:21,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:23,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:24,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:05:24,401 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:05:25,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:26,751 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:05:27,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:29,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:31,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:33,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:35,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:37,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:39,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:39,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:05:39,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:05:41,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:43,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:45,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:47,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:49,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:51,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:53,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:55,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:55,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:05:55,834 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:05:56,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:05:57,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:05:59,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:01,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:03,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:05,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:07,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:09,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:11,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:11,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:06:11,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:06:13,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:15,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:17,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:19,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:21,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:23,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:25,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:26,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:06:26,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:06:27,015 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:06:27,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:29,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:31,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:33,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:35,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:37,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:39,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:41,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:06:42,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:06:43,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:45,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:48,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:50,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:52,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:54,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:56,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:06:56,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:06:57,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:06:57,387 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:06:58,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:00,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:02,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:04,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:06,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:08,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:10,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:12,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:12,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:07:12,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:07:14,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:16,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:18,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:20,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:22,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:24,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:26,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:27,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:07:28,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:07:28,065 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:07:28,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:30,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:32,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:34,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:36,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:38,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:40,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:42,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:07:43,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:07:44,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:46,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:48,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:50,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:52,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:54,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:56,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:07:57,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:07:58,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:07:58,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:07:59,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:01,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:03,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:05,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:07,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:09,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:11,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:13,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:14,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:08:14,022 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:08:15,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:17,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:19,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:21,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:23,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:25,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:27,151 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:08:27,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:29,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:08:29,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:08:29,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:31,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:33,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:35,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:37,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:39,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:41,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:43,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:44,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:08:44,542 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:08:45,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:47,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:49,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:51,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:53,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:55,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:57,227 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:08:57,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:59,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:08:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:08:59,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:09:01,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:03,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:05,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:07,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:09,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:11,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:13,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:09:15,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:09:15,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:17,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:19,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:21,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:23,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:25,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:27,303 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:09:27,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:29,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:30,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:09:30,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:09:31,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:33,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:35,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:37,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:39,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:41,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:43,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:45,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:45,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:09:45,694 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:09:47,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:50,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:52,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:54,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:56,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:09:57,377 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:09:58,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:00,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:00,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:10:00,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:10:02,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:04,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:06,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:08,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:10,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:12,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:14,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:16,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:10:16,465 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:10:16,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:18,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:20,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:22,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:24,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:26,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:27,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:10:28,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:30,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:10:31,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:10:32,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:34,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:36,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:38,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:40,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:42,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:44,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:46,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:47,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:10:47,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:10:48,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:51,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:53,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:55,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:57,519 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:10:57,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:10:59,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:01,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:02,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:11:02,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:11:03,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:05,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:07,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:09,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:11,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:13,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:15,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:17,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:17,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:11:17,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:11:19,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:21,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:23,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:25,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:27,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:11:27,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:29,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:31,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:32,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:11:32,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:11:33,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:35,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:37,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:39,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:41,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:43,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:45,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:47,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:48,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:11:48,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:11:49,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:51,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:53,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:55,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:57,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:11:57,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:11:59,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:01,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:03,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:03,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:12:03,629 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:12:05,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:07,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:09,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:11,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:13,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:15,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:17,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:18,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:12:18,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:12:19,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:21,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:23,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:25,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:27,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:27,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:12:29,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:31,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:33,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:34,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:12:34,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:12:35,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:37,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:39,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:41,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:43,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:45,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:47,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:49,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:12:49,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:12:49,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:51,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:53,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:55,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:12:57,793 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:12:58,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:00,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:02,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:04,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:04,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:13:04,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:13:06,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:08,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:10,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:12,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:14,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:16,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:18,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:13:20,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:13:20,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:22,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:24,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:26,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:27,870 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:13:28,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:30,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:32,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:34,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:35,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:13:35,428 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:13:36,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:38,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:40,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:42,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:44,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:46,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:48,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:50,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:50,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:13:50,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:13:52,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:54,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:56,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:13:57,945 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:13:58,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:00,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:02,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:04,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:14:06,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:14:06,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:08,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:10,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:12,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:14,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:16,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:18,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:20,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:21,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:14:21,293 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:14:22,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:24,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:26,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:28,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:14:28,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:30,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:32,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:34,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:36,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:14:36,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:14:36,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:38,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:40,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:42,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:44,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:46,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:49,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:51,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:14:51,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:14:53,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:55,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:57,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:14:58,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:14:59,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:01,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:03,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:05,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:15:07,095 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:15:07,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:09,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:11,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:13,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:15,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:17,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:19,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:21,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:22,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:15:22,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:15:23,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:25,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:27,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:28,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:15:29,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:31,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:33,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:35,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:37,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:15:37,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:15:39,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:41,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:43,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:45,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:47,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:49,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:51,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:52,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:15:52,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:15:53,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:55,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:15:58,251 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:15:58,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:00,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:02,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:04,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:06,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:08,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:16:08,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:16:08,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:10,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:12,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:14,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:16,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:18,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:20,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:22,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:23,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:16:23,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:16:24,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:26,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:28,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:16:28,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:30,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:32,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:34,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:36,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:38,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:38,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:16:38,864 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:16:40,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:42,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:44,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:46,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:48,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:50,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:52,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:54,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:16:54,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:16:54,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:56,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:16:58,403 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:16:58,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:00,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:02,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:04,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:06,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:08,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:09,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:17:09,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:17:10,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:12,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:14,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:16,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:18,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:20,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:22,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:24,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:24,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:17:24,754 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:17:26,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:28,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:17:28,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:30,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:33,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:35,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:37,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:39,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:40,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:17:40,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:17:41,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:43,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:45,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:47,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:49,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:51,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:53,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:55,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:17:55,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:17:55,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:57,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:17:58,553 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:17:59,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:01,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:03,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:05,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:07,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:09,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:10,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:18:10,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:18:11,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:13,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:15,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:17,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:19,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:21,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:23,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:25,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:18:25,901 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:18:27,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:28,630 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:18:29,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:31,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:33,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:35,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:37,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:39,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:41,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:18:41,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:18:41,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:43,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:45,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:47,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:49,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:51,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:54,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:56,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:18:56,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:18:56,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:18:58,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:18:58,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:00,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:02,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:04,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:06,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:08,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:10,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:19:11,855 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:19:12,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:14,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:16,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:18,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:20,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:22,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:24,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:26,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:27,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:19:27,138 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:19:28,781 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:19:28,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:30,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:32,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:34,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:36,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:38,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:40,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:19:42,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:19:42,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:44,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:46,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:48,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:50,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:52,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:54,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:56,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:57,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:19:57,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:19:58,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:19:58,854 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:20:00,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:03,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:05,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:07,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:09,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:11,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:12,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:20:12,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:20:13,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:15,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:17,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:19,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:21,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:23,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:25,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:27,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:28,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:20:28,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:20:28,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:20:29,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:31,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:33,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:35,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:37,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:39,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:41,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:43,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:20:43,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:20:43,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:45,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:47,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:49,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:51,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:53,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:55,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:57,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:20:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:20:58,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:20:59,017 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:20:59,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:01,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:03,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:05,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:07,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:09,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:11,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:13,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:14,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:21:14,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:21:15,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:17,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:19,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:21,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:23,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:25,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:27,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:29,066 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:21:29,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:21:29,245 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:21:29,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:31,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:33,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:35,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:38,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:40,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:42,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:21:44,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:21:44,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:46,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:48,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:50,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:52,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:54,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:56,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:58,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:21:59,137 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:21:59,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:21:59,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:22:00,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:02,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:04,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:06,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:08,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:10,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:12,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:14,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:15,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:22:15,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:22:16,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:18,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:20,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:22,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:24,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:26,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:28,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:29,200 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:22:30,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:22:30,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:22:30,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:32,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:34,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:36,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:38,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:40,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:42,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:44,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:45,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:22:45,801 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:22:46,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:48,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:50,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:52,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:54,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:56,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:58,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:22:59,259 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:23:00,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:01,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:23:01,074 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:23:02,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:04,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:06,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:08,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:10,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:12,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:14,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:16,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:23:16,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:23:16,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:18,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:20,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:22,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:24,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:26,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:28,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:29,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:23:30,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:23:31,587 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:23:32,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:34,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:36,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:38,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:40,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:43,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:45,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:46,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:23:46,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:23:47,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:49,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:51,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:53,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:55,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:57,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:23:59,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:23:59,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:01,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:02,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:24:02,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:24:03,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:05,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:07,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:09,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:11,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:13,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:15,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:17,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:24:17,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:24:17,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:19,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:21,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:23,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:25,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:27,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:29,480 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:24:29,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:31,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:32,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:24:32,617 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:24:33,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:35,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:37,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:39,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:41,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:43,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:45,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:24:47,881 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:24:47,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:49,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:51,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:53,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:55,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:57,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:24:59,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:24:59,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:01,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:03,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:25:03,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:25:03,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:05,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:07,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:09,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:11,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:13,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:15,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:17,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:18,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:25:18,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:25:19,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:21,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:23,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:25,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:27,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:29,633 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:25:29,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:31,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:33,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:25:33,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:25:33,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:35,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:37,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:39,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:41,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:43,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:45,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:48,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:49,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:25:49,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:25:50,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:52,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:54,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:56,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:58,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:25:59,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:26:00,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:02,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:26:04,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:26:04,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:06,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:08,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:10,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:12,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:14,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:16,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:18,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:26:19,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:26:20,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:22,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:24,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:26,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:28,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:29,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:26:30,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:32,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:34,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:26:34,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:26:34,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:36,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:38,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:40,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:42,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:44,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:46,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:48,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:50,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:26:50,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:26:50,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:52,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:55,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:57,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:26:59,855 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:26:59,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:01,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:03,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:27:05,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:27:05,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:07,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:09,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:11,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:13,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:15,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:17,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:19,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:27:20,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:27:21,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:23,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:25,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:27,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:29,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:27:29,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:31,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:33,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:35,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:27:35,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:27:35,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:37,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:39,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:41,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:43,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:46,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:48,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:50,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:51,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:27:51,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:27:52,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:54,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:56,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:27:58,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:00,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:00,008 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:28:02,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:04,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:06,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:06,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:28:06,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:28:08,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:10,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:13,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:15,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:17,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:19,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:21,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:28:21,797 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:28:23,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:25,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:27,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:29,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:30,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:28:31,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:33,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:35,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:37,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:37,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:28:37,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:28:39,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:41,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:43,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:45,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:47,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:28:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:28:52,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:29:00,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:29:07,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:29:07,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:29:22,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:29:22,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:29:30,238 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:29:38,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:29:38,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:29:45,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:29:45,957 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 16:29:46,011 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 16:29:46,089 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 16:29:47,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 16:29:47,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:29:53,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:29:53,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:29:53,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:30:00,331 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:30:08,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:30:08,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:30:11,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:30:23,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:30:23,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:30:30,414 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:30:39,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:30:39,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:30:54,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:30:54,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:31:00,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:31:09,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:31:09,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:31:25,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:31:25,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:31:30,579 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:31:40,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:31:40,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:31:55,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:31:55,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:32:00,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:32:11,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:32:11,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:32:26,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:32:26,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:32:30,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:32:41,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:32:41,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:32:56,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:32:56,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:33:00,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:33:11,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:33:11,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:33:27,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:33:27,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:33:30,881 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:33:42,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:33:42,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:33:57,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:33:57,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:34:00,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:34:09,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:11,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:12,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:12,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:34:12,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:34:13,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:15,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:16,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:16,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:17,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:18,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:19,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:21,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:22,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:23,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:24,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:24,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:28,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:34:28,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:34:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:30,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:31,034 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:34:32,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:32,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:34,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:34,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:36,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:36,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:37,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:38,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:40,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:42,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:42,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:42,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:43,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:34:43,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:34:44,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:46,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:46,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:46,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:47,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:48,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:50,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:52,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:52,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:52,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:54,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:55,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:56,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:57,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:34:58,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:34:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:34:58,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:35:00,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:01,101 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:35:08,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:11,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:14,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:35:14,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:35:14,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:18,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:19,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:22,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:23,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:23,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:23,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:26,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:28,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:29,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:35:29,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:35:31,170 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:35:32,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:36,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:38,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:39,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:41,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:42,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:42,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:44,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:44,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:35:44,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:35:46,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:46,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:50,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:52,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:54,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:54,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:56,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:35:56,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:56,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:59,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:35:59,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:35:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:59,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:35:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:00,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:00,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,246 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:36:01,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:02,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:04,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:06,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:07,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:08,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:10,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:12,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:12,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:12,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:14,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:36:15,127 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:36:16,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:16,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:18,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:19,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:22,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:24,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:26,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:26,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:28,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:29,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:30,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:30,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:36:30,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:36:31,318 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:36:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:31,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:32,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:33,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:33,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:34,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:35,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:35,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:36,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:37,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:39,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:39,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:39,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:41,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:43,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:43,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:43,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:45,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:45,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:36:45,725 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:36:46,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:46,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:47,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:48,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:36:49,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:51,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:36:53,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:00,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:01,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:37:01,089 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:37:01,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:01,396 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:37:07,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:11,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:11,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:11,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:15,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:16,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:37:16,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:37:17,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:18,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:18,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:21,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:22,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:22,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:25,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:27,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:29,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:31,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:31,470 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:37:31,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:37:31,738 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:37:35,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:35,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:37,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:37,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:37,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:38,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:39,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:40,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:40,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:43,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:43,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:45,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:46,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:37:46,979 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:37:47,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:47,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:51,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:52,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:52,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:53,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:53,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:54,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:54,131 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 16:37:54,346 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 16:37:54,423 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 16:37:55,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 16:37:55,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:56,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:57,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:37:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:57,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:37:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:01,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:01,570 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:38:02,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:38:02,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:38:03,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:05,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:07,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:08,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:08,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:09,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:11,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:12,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:12,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:13,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:15,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:17,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:17,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:38:17,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:38:18,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:18,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:19,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:21,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:22,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:22,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:23,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:25,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:26,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:27,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:28,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:28,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:29,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:30,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:31,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:31,639 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:38:32,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:32,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:38:32,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:38:33,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:33,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:33,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:35,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:36,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:37,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:39,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:40,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:40,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:41,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:43,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:45,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:47,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:38:48,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:38:49,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:49,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:53,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:38:59,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:38:59,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:00,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:01,712 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:39:03,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:39:03,335 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:39:04,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:08,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:09,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:09,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:12,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:16,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:18,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:18,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:18,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:39:18,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:39:19,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:19,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:22,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:23,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:26,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:28,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:30,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:30,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:31,786 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:39:31,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:31,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:31,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:31,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:32,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:39:33,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:39:34,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:36,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:36,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:36,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:39,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:40,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:42,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:43,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:44,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:46,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:46,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:47,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:48,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:39:49,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:39:50,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:52,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:52,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:52,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:54,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:56,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:39:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:39:57,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:00,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:00,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:01,861 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:40:02,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:04,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:04,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:40:04,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:40:06,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:07,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:08,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:10,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:11,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:12,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:13,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:14,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:15,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:16,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:18,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:19,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:40:19,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:40:19,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:19,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:20,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:20,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:22,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:22,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:24,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:24,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:25,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:25,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:25,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:25,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:26,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:28,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:29,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:29,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:30,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:31,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:40:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:32,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:33,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:34,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:35,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:40:35,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:40:36,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:37,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:37,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:38,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:38,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:39,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:40,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:44,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:44,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:48,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:50,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:40:50,344 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:40:50,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:53,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:53,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:40:56,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:40:56,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:00,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:01,995 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:41:04,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:05,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:41:05,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:41:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:07,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:07,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:08,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:12,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:14,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:14,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:14,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:15,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:18,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:41:20,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:41:23,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:24,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:24,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:25,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:27,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:29,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:30,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:30,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:31,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:32,069 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:41:33,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:36,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:41:36,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:41:37,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:38,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:38,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:39,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:41,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:41,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:42,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:42,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:42,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:42,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:42,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:42,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:44,725 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 16:41:44,942 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 16:41:45,010 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 16:41:45,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:45,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 16:41:45,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:47,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:48,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:49,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:51,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:41:51,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:41:51,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:52,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:52,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:53,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:54,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:54,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:55,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:57,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:41:59,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:41:59,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:01,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:02,158 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:42:03,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:05,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:05,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:06,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:42:06,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:42:07,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:09,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:09,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:10,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:10,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:11,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:11,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:11,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:12,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:13,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:14,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:15,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:15,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:17,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:17,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:18,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:19,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:20,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:21,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:22,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:42:22,077 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:42:22,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:22,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:23,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:25,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:26,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:26,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:27,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:28,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:29,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:30,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:30,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:31,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:31,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:32,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:42:33,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:34,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:35,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:37,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:42:37,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:42:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:41,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:45,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:49,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:49,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:49,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:52,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:42:52,537 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:42:53,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:42:54,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:54,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:42:57,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:01,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:01,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:02,304 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:43:05,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:07,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:43:07,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:43:08,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:08,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:09,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:13,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:13,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:13,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:15,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:19,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:21,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:22,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:22,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:23,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:43:23,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:43:25,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:26,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:26,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:27,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:29,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:29,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:30,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:31,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:32,375 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:43:32,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:32,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:33,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:35,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:37,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:38,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:43:38,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:43:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:38,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:38,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:38,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:41,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:42,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:42,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:43,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:45,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:47,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:48,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:49,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:52,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:52,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:43:53,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:43:54,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:56,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:43:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:43:58,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:00,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:02,450 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:44:02,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:03,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:04,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:06,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:07,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:08,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:08,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:44:08,962 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:44:09,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:10,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:10,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:12,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:14,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:15,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:15,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:16,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:17,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:18,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:18,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:19,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:20,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:22,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:22,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:24,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:44:24,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:44:24,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:24,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:26,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:28,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:30,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:32,526 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:44:38,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:39,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:44:39,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:44:40,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:46,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:49,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:50,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:52,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:44:53,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:53,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:44:54,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:44:54,996 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:44:56,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:00,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:01,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:01,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:02,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:02,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:45:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:04,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:06,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:08,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:10,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:45:10,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:45:12,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:13,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:14,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:18,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:20,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:20,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:22,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:22,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:24,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:24,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:24,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:45:25,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:45:25,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:25,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:27,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:29,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:29,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:30,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,071 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 16:45:31,276 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 16:45:31,345 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 16:45:31,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 16:45:31,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:31,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:31,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:32,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:32,682 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:45:33,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:35,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:36,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:37,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:37,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:38,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:39,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:40,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:40,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:45:40,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:45:41,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:41,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:41,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:43,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:44,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:45,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:47,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:49,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:49,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:51,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:52,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:52,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:53,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:54,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:54,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:55,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:55,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:45:55,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:45:57,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:45:59,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:45:59,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:00,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:01,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:02,750 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:46:03,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:04,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:04,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:05,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:07,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:11,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:46:11,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:46:15,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:15,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:17,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:23,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:24,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:24,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:26,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:46:26,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:46:27,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:31,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:31,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:32,824 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:46:35,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:38,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:39,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:41,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:41,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:41,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:46:41,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:46:45,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:49,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:51,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:53,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:46:56,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:46:57,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:46:57,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:57,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:46:59,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:01,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:02,890 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:47:04,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:05,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:07,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:08,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:09,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:11,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:11,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:12,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:47:12,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:47:13,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:13,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:15,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:17,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:17,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:19,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:20,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:20,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:21,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:23,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:25,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:26,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:26,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:47:27,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:47:27,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:27,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:27,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:29,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:31,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:31,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:32,964 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:47:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:33,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:35,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:36,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:37,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:38,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:39,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:40,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:41,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:41,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:42,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:47:42,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:47:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:43,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:44,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:44,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:45,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:47,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:49,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:51,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:51,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:51,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:53,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:55,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:55,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:56,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:56,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:47:57,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:47:58,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:47:58,080 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:47:59,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:00,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:01,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:03,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:48:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:10,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:11,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:13,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:48:13,323 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:48:16,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:16,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:18,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:20,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:24,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:25,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:28,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:28,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:48:28,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:48:29,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:29,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:32,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:33,095 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:48:33,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:33,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:34,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:38,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:39,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:39,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:40,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:42,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:42,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:43,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:48:43,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:48:44,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:46,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:50,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:52,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:52,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:52,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:56,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:57,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:48:58,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:48:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:48:59,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:49:00,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:00,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:02,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:02,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:03,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:49:04,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:05,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:05,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:06,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:08,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:10,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:12,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:13,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:13,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:49:14,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:49:15,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:16,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:18,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:19,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:20,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:21,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:21,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:21,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:21,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:22,555 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 16:49:22,747 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 16:49:22,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:22,812 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 16:49:23,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:23,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 16:49:24,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:24,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:24,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:25,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:26,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:27,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:28,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:29,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:49:29,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:49:30,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:32,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:32,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:33,255 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:49:34,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:34,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:36,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:38,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:38,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:39,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:39,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:40,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:42,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:44,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:44,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:49:44,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:49:46,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:47,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:47,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:48,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:49,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:49,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:50,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:51,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:52,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:52,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:53,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:49:54,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:49:56,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:00,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:50:00,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:50:03,329 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:50:03,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:03,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:06,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:07,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:07,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:12,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:15,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:50:15,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:50:16,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:18,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:20,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:23,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:23,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:24,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:26,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:26,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:28,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:29,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:30,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:50:30,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:50:31,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:33,404 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:50:35,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:39,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:41,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:41,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:45,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:46,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:50:46,190 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:50:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:47,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:51,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:53,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:50:55,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:50:59,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:00,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:00,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:51:01,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:51:01,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:01,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:02,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:03,473 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:51:03,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:05,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:07,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:09,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:11,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:13,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:15,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:15,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:16,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:51:16,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:51:17,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:17,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:19,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:19,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:21,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:22,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:22,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:23,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:25,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:27,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:27,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:27,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:29,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:31,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:51:32,028 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:51:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:32,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,546 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:51:33,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:33,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:35,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:36,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:36,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:37,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:38,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:38,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:39,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:41,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:42,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:43,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:45,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:47,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:51:47,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:51:47,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:48,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:49,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:50,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:51:51,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:51,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:51:53,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:02,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:52:02,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:52:02,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:02,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:03,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:52:04,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:05,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:09,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:10,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:10,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:12,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:13,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:14,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:17,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:18,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:18,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:52:18,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:52:21,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:23,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:23,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:25,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:29,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:30,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:30,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:31,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:33,692 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:52:34,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:52:34,203 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:52:35,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:35,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:37,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:39,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:39,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:41,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:43,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:47,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:47,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:48,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:48,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:49,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:52:49,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:52:49,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:53,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:55,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:52:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:52:58,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:00,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:01,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:01,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:03,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:53:04,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:53:04,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:53:04,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:05,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:05,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:06,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:08,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:08,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:10,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:12,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:14,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:14,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:16,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:17,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 16:53:17,718 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 16:53:17,936 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 16:53:17,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:18,005 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 16:53:18,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 16:53:19,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:53:20,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:53:20,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:22,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:24,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:26,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:28,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:30,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:32,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:34,045 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:53:34,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:35,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:53:35,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:53:36,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:38,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:40,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:42,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:44,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:46,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:48,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:50,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:53:50,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:53:50,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:52,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:54,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:56,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:53:58,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:00,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:02,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:04,119 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:54:04,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:54:05,772 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:54:06,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:08,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:10,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:12,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:14,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:16,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:18,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:20,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:21,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:54:21,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:54:22,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:24,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:26,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:28,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:31,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:33,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:34,193 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:54:35,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:36,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:54:36,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:54:37,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:39,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:41,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:43,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:45,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:47,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:49,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:51,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:54:51,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:54:51,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:53,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:55,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:57,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:54:59,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:01,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:03,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:04,268 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:55:05,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:07,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:55:07,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:55:07,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:09,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:11,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:13,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:15,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:17,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:19,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:21,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:22,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:55:22,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:55:23,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:26,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:28,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:30,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:32,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:34,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:34,335 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:55:36,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:37,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:55:37,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:55:38,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:41,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:43,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:45,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:47,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:49,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:51,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:52,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:55:52,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:55:53,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:55,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:57,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:55:59,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:01,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:03,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:04,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:56:05,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:07,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:56:08,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:56:09,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:11,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:13,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:15,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:17,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:19,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:21,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:23,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:23,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:56:23,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:56:25,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:27,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:29,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:31,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:33,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:34,486 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:56:35,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:37,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:38,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:56:38,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:56:39,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:41,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:43,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:45,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:47,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:49,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:51,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:53,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:53,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:56:53,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:56:55,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:57,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:56:59,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:01,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:03,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:04,563 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:57:05,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:07,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:09,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:57:09,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:57:11,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:13,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:16,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:18,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:20,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:22,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:24,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:24,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:57:24,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:57:26,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:28,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:30,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:32,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:34,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:34,640 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:57:36,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:38,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:39,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:57:39,956 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:57:40,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:42,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:44,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:46,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:48,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:50,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:52,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:54,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:55,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:57:55,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:57:56,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:57:58,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:00,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:02,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:04,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:04,717 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:58:06,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:08,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:10,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:10,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:58:10,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:58:12,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:14,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:16,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:18,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:21,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:23,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:25,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:25,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:58:25,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:58:27,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:29,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:31,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:33,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:34,796 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:58:35,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:37,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:39,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:41,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:41,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:58:41,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:58:43,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:45,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:47,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:49,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:51,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:53,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:55,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:56,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:58:56,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:58:57,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:58:59,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:01,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:03,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:04,874 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:59:05,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:07,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:09,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:11,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:11,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:59:11,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:59:13,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:15,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:17,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:19,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:21,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:23,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:25,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:27,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:27,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:59:27,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:59:29,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:31,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:33,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:34,937 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 16:59:35,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:37,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:39,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:41,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:42,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:59:42,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:59:43,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:45,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:47,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:49,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:51,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:53,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:55,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:57,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 16:59:57,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 16:59:57,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 16:59:59,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:01,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:03,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:05,008 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:00:05,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:08,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:10,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:12,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:13,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:00:13,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:00:14,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:16,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:18,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:20,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:22,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:24,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:26,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:28,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:00:28,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:00:30,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:32,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:34,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:35,082 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:00:36,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:38,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:40,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:42,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:43,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:00:43,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:00:44,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:46,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:48,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:50,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:52,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:54,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:56,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:58,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:00:59,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:00:59,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:01:00,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:02,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:04,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:05,156 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:01:06,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:08,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:10,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:12,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:14,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:14,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:01:14,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:01:16,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:18,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:20,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:22,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:24,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:26,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:28,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:29,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:01:29,709 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:01:30,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:32,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:34,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:35,229 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:01:36,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:39,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:41,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:43,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:45,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:01:45,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:01:45,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:47,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:49,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:51,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:53,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:55,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:57,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:01:59,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:00,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:02:00,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:02:01,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:03,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:05,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:05,301 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:02:07,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:09,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:11,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:13,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:15,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:15,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:02:15,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:02:17,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:19,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:21,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:23,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:25,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:27,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:29,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:31,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:02:31,091 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:02:31,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:33,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:35,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:35,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:02:37,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:39,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:41,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:43,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:45,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:46,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:02:46,323 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:02:47,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:49,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:51,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:53,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:55,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:57,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:02:59,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:01,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:03:01,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:03:02,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:04,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:05,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:03:06,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:08,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:10,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:12,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:14,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:16,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:16,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:03:16,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:03:18,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:20,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:22,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:24,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:26,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:28,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:30,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:32,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:03:32,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:03:32,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:34,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:35,524 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:03:36,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:38,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:40,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:42,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:44,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:46,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:47,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:03:47,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:03:48,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:50,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:52,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:54,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:56,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:03:58,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:00,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:02,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:02,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:04:02,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:04:04,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:05,600 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:04:06,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:08,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:10,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:12,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:14,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:16,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:18,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:04:18,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:04:18,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:20,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:22,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:24,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:26,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:28,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:30,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:32,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:04:33,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:04:34,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:35,679 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:04:36,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:38,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:40,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:42,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:44,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:47,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:48,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:04:48,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:04:49,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:51,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:53,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:55,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:57,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:04:59,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:01,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:03,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:03,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:05:03,968 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:05:05,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:05,754 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:05:07,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:09,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:11,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:13,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:15,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:17,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:19,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:05:19,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:05:19,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:21,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:23,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:25,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:27,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:29,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:31,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:33,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:05:34,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:05:35,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:35,828 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:05:37,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:39,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:41,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:43,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:45,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:47,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:49,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:49,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:05:49,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:05:51,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:53,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:55,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:57,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:05:59,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:01,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:03,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:04,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:06:04,933 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:06:05,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:05,904 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:06:07,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:10,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:12,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:14,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:16,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:18,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:20,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:20,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:06:20,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:06:22,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:24,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:26,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:28,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:30,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:32,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:34,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:35,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:06:35,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:06:35,981 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:06:36,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:38,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:40,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:42,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:44,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:46,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:48,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:50,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:50,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:06:50,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:06:52,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:54,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:56,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:06:58,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:00,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:02,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:04,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:06,057 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:07:06,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:07:06,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:07:06,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:08,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:10,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:12,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:14,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:16,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:18,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:20,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:21,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:07:21,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:07:22,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:24,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:26,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:28,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:30,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:32,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:34,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:36,133 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:07:36,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:36,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:07:36,731 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:07:38,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:40,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:42,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:44,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:46,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:49,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:51,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:51,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:07:51,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:07:53,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:55,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:57,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:07:59,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:01,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:03,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:05,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:06,207 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:08:07,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:08:07,255 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:08:07,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:09,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:11,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:13,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:15,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:17,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:19,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:21,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:22,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:08:22,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:08:23,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:25,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:27,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:29,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:31,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:33,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:35,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:36,284 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:08:37,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:37,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:08:37,890 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:08:39,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:41,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:43,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:45,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:47,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:49,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:51,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:08:53,125 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:08:53,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:55,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:57,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:08:59,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:01,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:03,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:05,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:06,360 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:09:07,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:08,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:09:08,376 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:09:09,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:11,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:13,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:15,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:17,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:20,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:22,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:23,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:09:23,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:09:24,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:26,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:28,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:30,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:32,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:34,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:36,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:36,433 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:09:38,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:39,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:09:39,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:09:40,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:42,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:44,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:46,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:48,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:50,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:52,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:54,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:09:54,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:09:56,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:09:58,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:00,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:02,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:04,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:06,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:06,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:10:08,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:10:10,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:10:10,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:12,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:14,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:16,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:18,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:20,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:22,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:24,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:25,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:10:25,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:10:26,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:28,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:30,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:33,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:35,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:36,575 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:10:37,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:39,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:40,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:10:40,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:10:41,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:43,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:45,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:47,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:49,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:51,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:53,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:55,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:55,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:10:55,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:10:57,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:10:59,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:01,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:03,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:05,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:06,651 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:11:07,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:09,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:11,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:11:11,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:11:11,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:13,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:15,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:17,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:19,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:21,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:23,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:25,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:26,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:11:26,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:11:27,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:29,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:31,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:33,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:35,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:36,728 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:11:37,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:39,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:41,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:41,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:11:41,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:11:43,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:45,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:47,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:49,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:51,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:53,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:55,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:57,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:11:57,416 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:11:57,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:11:59,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:01,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:04,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:06,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:06,803 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:12:08,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:10,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:12,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:12,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:12:12,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:12:14,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:16,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:18,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:20,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:22,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:24,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:26,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:28,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:28,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:12:28,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:12:30,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:32,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:34,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:36,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:36,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:12:38,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:40,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:42,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:43,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:12:43,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:12:44,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:46,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:48,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:50,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:52,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:54,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:56,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:58,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:12:59,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:12:59,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:13:00,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:02,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:04,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:06,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:06,953 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:13:08,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:10,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:12,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:14,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:14,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:13:14,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:13:16,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:18,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:20,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:22,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:24,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:26,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:29,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:13:30,289 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:13:31,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:33,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:35,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:37,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:13:37,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:39,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:41,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:43,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:45,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:13:45,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:13:47,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:49,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:51,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:53,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:55,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:57,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:13:59,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:01,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:01,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:14:01,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:14:03,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:05,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:07,105 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:14:07,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:09,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:11,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:13,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:15,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:14:16,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:14:17,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:19,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:21,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:23,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:25,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:27,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:29,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:31,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:14:32,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:14:33,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:35,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:37,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:14:37,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:39,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:41,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:43,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:45,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:47,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:47,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:14:47,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:14:49,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:52,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:54,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:56,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:14:58,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:00,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:02,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:15:02,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:15:04,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:06,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:07,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:15:08,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:10,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:12,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:14,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:16,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:15:18,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:15:18,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:20,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:22,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:24,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:26,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:28,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:30,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:32,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:33,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:15:33,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:15:34,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:36,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:37,340 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:15:38,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:40,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:42,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:44,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:46,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:48,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:48,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:15:48,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:15:50,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:52,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:54,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:56,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:15:58,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:00,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:02,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:04,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:16:04,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:16:04,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:06,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:07,412 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:16:08,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:10,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:12,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:14,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:16,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:18,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:19,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:16:19,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:16:20,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:23,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:25,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:27,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:29,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:31,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:33,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:34,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:16:34,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:16:35,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:37,490 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:16:37,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:39,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:41,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:43,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:45,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:47,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:49,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:50,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:16:50,077 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:16:51,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:53,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:55,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:57,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:16:59,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:01,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:03,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:17:05,543 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:17:05,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:07,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:07,567 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:17:09,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:11,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:13,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:15,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:17,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:19,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:20,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:17:20,969 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:17:21,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:23,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:25,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:27,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:29,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:32,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:34,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:17:36,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:17:36,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:37,642 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:17:38,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:40,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:42,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:44,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:46,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:48,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:50,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:52,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:17:52,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:17:52,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:54,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:56,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:17:58,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:00,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:02,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:04,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:06,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:07,716 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:18:07,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:18:07,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:18:08,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:10,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:12,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:14,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:16,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:18,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:20,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:22,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:18:23,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:18:24,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:26,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:28,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:30,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:32,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:34,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:36,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:37,790 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:18:38,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:38,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:18:38,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:18:40,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:42,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:44,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:46,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:48,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:50,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:52,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:54,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:55,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:18:55,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:18:56,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:18:58,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:00,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:02,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:05,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:07,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:07,864 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:19:09,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:19:11,158 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:19:11,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:13,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:15,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:17,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:19,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:21,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:23,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:25,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:26,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:19:26,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:19:27,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:29,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:31,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:33,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:35,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:37,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:37,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:19:39,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:41,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:42,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:19:42,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:19:43,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:45,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:47,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:49,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:51,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:53,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:55,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:57,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:19:57,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:19:57,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:19:59,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:01,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:03,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:05,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:07,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:08,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:20:09,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:11,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:13,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:20:13,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:20:13,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:15,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:17,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:19,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:21,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:23,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:25,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:27,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:28,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:20:28,492 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:20:30,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:32,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:34,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:36,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:38,080 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:20:38,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:40,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:42,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:43,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:20:43,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:20:44,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:46,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:48,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:50,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:52,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:54,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:56,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:58,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:20:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:20:59,740 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:21:00,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:02,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:04,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:06,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:08,156 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:21:08,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:10,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:12,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:14,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:21:15,116 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:21:16,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:18,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:20,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:22,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:24,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:26,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:28,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:30,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:21:30,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:21:30,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:32,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:34,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:36,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:38,236 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:21:38,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:40,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:42,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:44,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:45,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:21:45,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:21:46,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:48,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:50,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:52,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:54,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:56,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:21:58,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:00,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:01,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:22:01,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:22:02,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:04,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:06,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:08,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:22:08,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:10,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:12,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:15,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:17,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:22:17,375 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:22:17,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:19,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:21,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:23,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:25,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:27,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:29,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:31,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:22:32,876 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:22:33,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:35,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:37,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:38,396 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:22:39,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:41,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:43,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:45,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:47,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:48,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:22:48,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:22:49,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:51,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:53,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:55,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:57,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:22:59,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:01,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:03,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:03,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:23:03,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:23:05,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:07,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:08,475 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:23:09,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:11,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:13,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:15,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:17,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:19,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:23:19,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:23:19,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:21,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:23,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:25,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:27,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:29,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:31,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:33,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:34,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:23:34,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:23:36,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:38,553 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:23:38,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:40,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:42,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:44,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:46,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:48,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:49,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:23:49,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:23:50,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:52,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:54,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:56,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:23:58,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:00,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:02,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:04,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:05,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:24:05,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:24:06,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:08,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:24:08,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:10,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:12,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:14,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:16,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:18,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:24:20,488 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:24:20,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:22,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:24,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:26,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:28,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:30,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:32,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:34,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:35,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:24:35,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:24:36,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:38,706 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:24:38,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:40,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:42,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:44,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:46,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:48,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:50,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:51,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:24:51,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:24:52,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:54,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:56,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:24:58,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:00,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:02,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:04,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:06,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:25:06,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:25:06,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:08,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:08,782 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:25:10,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:12,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:14,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:16,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:18,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:20,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:21,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:25:21,727 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:25:23,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:25,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:27,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:29,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:31,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:33,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:35,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:36,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:25:36,966 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:25:37,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:38,858 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:25:39,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:41,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:43,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:45,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:47,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:49,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:51,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:25:52,292 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:25:53,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:55,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:57,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:25:59,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:01,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:03,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:05,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:07,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:26:07,626 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:26:07,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:08,936 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:26:09,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:11,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:13,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:15,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:17,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:19,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:21,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:22,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:26:22,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:26:23,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:25,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:27,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:29,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:31,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:33,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:35,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:37,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:38,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:26:38,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:26:39,013 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:26:39,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:42,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:44,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:46,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:48,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:50,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:52,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:53,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:26:53,980 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:26:54,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:56,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:26:58,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:00,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:02,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:04,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:06,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:08,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:09,088 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:27:09,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:27:09,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:27:10,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:12,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:14,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:16,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:18,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:20,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:22,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:27:24,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:27:24,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:26,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:28,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:30,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:32,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:34,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:36,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:38,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:39,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:27:40,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:27:40,345 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:27:40,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:42,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:44,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:46,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:48,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:50,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:52,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:54,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:55,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:27:55,614 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:27:56,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:27:58,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:00,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:02,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:04,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:06,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:09,239 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:28:09,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:10,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:28:10,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:28:11,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:13,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:15,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:17,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:19,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:21,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:23,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:25,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:26,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:28:26,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:28:27,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:29,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:31,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:33,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:35,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:37,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:39,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:28:39,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:41,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:28:41,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:28:41,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:43,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:45,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:47,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:49,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:51,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:53,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:55,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:57,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:28:57,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:28:57,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:28:59,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:01,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:03,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:05,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:07,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:09,389 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:29:09,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:11,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:29:12,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:29:13,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:15,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:17,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:19,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:21,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:23,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:25,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:27,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:28,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:29:28,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:29:29,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:31,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:34,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:36,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:38,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:39,461 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:29:40,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:42,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:43,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:29:43,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:29:44,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:46,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:48,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:50,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:52,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:54,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:56,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:58,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:29:58,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:29:58,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:30:00,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:02,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:04,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:06,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:08,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:09,534 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:30:10,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:12,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:14,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:30:14,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:30:14,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:16,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:18,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:20,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:22,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:24,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:26,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:28,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:30:29,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:30:30,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:32,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:34,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:36,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:38,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:39,606 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:30:40,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:42,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:44,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:45,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:30:45,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:30:46,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:48,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:50,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:52,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:54,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:56,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:30:58,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:00,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:31:00,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:31:00,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:02,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:04,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:06,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:08,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:09,682 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:31:10,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:12,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:15,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:16,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:31:16,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:31:17,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:19,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:21,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:23,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:25,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:27,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:29,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:31,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:32,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:31:32,329 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:31:33,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:35,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:37,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:39,755 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:31:39,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:41,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:43,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:45,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:47,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:31:47,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:31:47,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:49,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:51,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:53,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:55,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:57,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:31:59,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:01,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:03,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:32:03,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:32:03,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:05,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:07,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:09,829 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:32:09,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:11,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:13,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:15,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:17,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:18,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:32:18,392 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:32:19,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:21,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:23,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:25,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:27,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:29,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:31,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:33,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:32:33,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:32:33,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:35,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:37,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:39,906 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:32:39,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:41,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:43,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:45,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:47,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:49,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:32:49,022 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:32:50,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:52,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:54,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:56,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:32:58,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:00,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:02,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:33:04,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:33:04,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:06,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:08,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:09,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:33:10,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:12,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:14,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:16,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:18,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:19,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:33:19,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:33:20,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:22,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:24,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:26,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:28,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:30,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:32,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:34,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:35,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:33:35,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:33:36,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:38,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:40,058 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:33:40,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:42,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:44,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:46,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:48,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:50,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:33:50,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:33:51,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:53,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:55,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:57,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:33:59,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:01,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:03,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:05,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:34:05,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:34:07,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:09,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:10,134 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:34:11,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:13,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:15,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:17,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:19,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:21,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:21,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:34:21,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:34:23,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:25,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:27,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:29,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:32,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:34,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:36,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:36,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:34:36,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:34:38,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:40,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:40,210 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:34:42,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:44,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:46,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:48,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:50,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:51,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:34:51,664 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:34:52,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:54,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:56,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:34:58,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:00,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:02,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:04,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:06,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:07,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:35:07,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:35:08,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:10,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:10,284 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:35:12,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:14,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:16,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:18,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:20,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:22,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:35:22,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:35:22,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:35:37,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:35:37,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:35:40,361 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:35:53,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:35:53,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:36:09,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:36:09,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:36:10,437 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:36:20,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:36:20,893 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 17:36:20,948 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 17:36:21,022 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 17:36:21,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 17:36:22,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:36:23,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:36:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:36:24,843 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:36:28,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:36:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:36:40,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:36:42,067 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:36:44,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:36:57,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:36:57,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:37:10,606 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:37:12,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:37:12,384 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:37:27,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:37:27,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:37:40,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:37:42,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:37:42,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:37:58,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:37:58,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:38:10,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:38:13,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:38:13,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:38:28,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:38:28,968 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:38:40,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:38:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:38:44,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:38:59,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:38:59,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:39:10,926 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:39:14,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:39:14,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:39:30,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:39:30,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:39:41,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:39:45,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:39:45,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:40:00,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:40:00,997 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:40:11,083 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:40:16,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:40:16,568 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:40:31,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:40:31,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:40:41,163 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:40:43,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:46,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:47,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:40:47,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:40:47,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:48,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:48,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:49,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:51,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:53,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:55,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:55,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:56,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:40:57,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:40:59,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:01,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:01,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:01,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:02,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:41:02,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:41:03,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:04,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:04,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:05,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:07,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:08,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:08,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:09,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:10,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:10,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:11,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:11,231 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:41:13,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:13,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:13,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:15,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:17,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:17,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:41:17,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:41:20,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:20,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:25,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:30,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:31,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:32,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:41:32,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:41:35,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:36,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:39,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:41,304 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:41:42,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:42,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:43,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:45,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:47,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:48,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:41:48,157 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:41:49,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:49,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:53,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:54,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:54,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:55,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:59,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:41:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:41:59,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:01,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:02,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:03,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:42:03,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:42:04,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:04,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:05,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:06,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:06,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:07,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:09,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:11,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:42:13,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:13,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:15,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:17,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:18,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:42:18,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:42:19,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:21,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:22,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:22,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:25,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:27,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:27,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:29,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:29,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:31,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:31,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:33,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:34,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:42:34,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:42:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:35,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:36,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:38,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:39,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:39,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:40,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:41,455 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:42:42,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:43,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:44,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:45,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:46,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:48,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:49,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:42:49,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:42:50,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:50,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:51,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:52,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:53,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:53,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:54,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:56,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:56,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:57,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:58,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:42:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:42:59,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:00,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:02,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:04,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:04,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:04,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:43:04,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:43:06,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:06,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:06,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:10,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:11,529 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:43:20,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:43:20,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:43:24,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:26,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:26,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:28,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:29,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:29,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:32,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:33,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:35,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:43:35,771 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:43:36,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:38,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:38,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:38,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:39,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:40,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:41,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:43:42,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:43,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:43,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:44,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:46,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:48,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:48,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:51,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:43:51,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:43:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:52,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:54,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:43:56,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:56,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:43:58,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:00,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:00,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:00,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:02,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:04,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:05,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:05,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:44:06,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:44:06,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:06,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:08,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:10,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:11,681 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:44:12,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:12,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:12,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:14,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:16,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:16,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:16,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:18,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:19,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:20,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:21,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:44:21,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:44:22,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:22,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:22,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:24,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:25,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:26,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:27,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:28,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:28,228 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 17:44:28,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:28,435 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 17:44:28,513 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 17:44:29,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 17:44:29,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:30,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:30,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:32,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:34,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:34,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:35,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:35,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:36,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:37,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:44:37,294 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:44:37,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:37,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:38,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:39,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:40,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:41,844 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:44:42,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:44,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:45,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:45,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:48,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:49,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:50,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:50,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:51,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:52,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:44:52,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:44:52,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:52,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:53,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:55,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:44:57,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:44:59,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:00,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:01,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:03,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:05,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:08,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:45:08,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:45:11,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:11,919 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:45:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:15,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:20,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:20,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:21,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:23,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:45:23,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:45:25,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:27,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:29,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:29,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:31,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:34,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:35,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:35,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:37,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:45:38,527 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:45:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:41,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:41,993 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:45:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:42,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:43,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:45,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:45,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:47,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:47,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:49,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:50,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:53,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:53,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:45:53,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:45:55,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:56,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:57,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:45:58,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:45:59,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:03,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:04,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:04,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:05,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:06,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:07,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:09,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:46:09,314 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:46:09,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:11,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:12,068 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:46:13,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:14,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:15,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:16,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:16,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:17,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:19,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:20,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:20,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:21,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:21,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:22,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:23,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:24,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:46:24,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:46:25,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:27,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:27,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:27,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:29,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:31,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:31,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:31,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:32,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:33,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:33,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:35,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:37,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:39,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:46:39,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:46:41,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:41,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:41,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:42,140 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:46:43,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:43,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:43,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:44,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:45,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:46,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:46,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:47,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:48,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:49,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:50,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:51,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:53,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:46:55,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:46:55,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:46:58,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:46:58,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:03,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:09,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:10,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:47:10,387 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:47:10,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:12,206 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:47:14,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:17,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:18,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:20,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:24,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:25,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:25,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:47:25,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:47:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:27,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:28,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:30,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:32,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:32,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:33,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:34,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:35,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:35,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:36,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:37,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:40,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:40,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:47:40,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:47:41,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:41,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:42,278 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:47:42,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:43,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:43,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:46,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:48,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:48,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:50,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:54,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:54,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:55,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:55,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:47:56,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:47:56,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:47:56,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:56,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:47:58,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:00,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:02,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:02,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:03,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:04,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:05,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:05,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:08,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:10,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:11,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:48:11,371 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:48:12,345 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:48:12,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:13,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:14,535 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 17:48:14,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:14,716 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 17:48:14,784 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 17:48:15,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 17:48:16,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:17,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:18,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:20,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:22,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:22,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:24,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:24,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:26,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:48:26,672 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:48:26,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:28,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:28,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:28,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:30,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:30,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:32,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:34,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:34,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:36,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:37,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:37,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:38,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:39,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:41,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:48:41,953 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:48:42,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:48:42,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:42,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:42,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:43,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:48:44,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:46,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:55,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:48:57,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:48:57,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:49:00,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:01,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:04,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:04,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:05,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:09,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:12,527 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:49:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:49:12,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:49:13,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:13,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:13,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:15,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:19,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:23,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:24,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:25,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:27,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:49:27,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:49:29,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:29,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:31,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:33,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:33,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:34,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:35,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:36,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:36,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:37,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:39,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:41,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:49:42,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:43,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:49:43,250 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:49:43,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:45,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:47,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:49,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:49,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:51,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:51,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:52,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:55,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:57,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:49:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:49:58,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:49:58,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:49:59,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:01,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:01,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:03,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:05,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:07,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:09,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:11,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:12,677 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:50:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:12,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:13,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:14,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:50:14,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:50:15,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:17,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:18,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:18,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:19,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:21,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:22,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:23,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:24,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:25,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:25,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:27,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:28,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:28,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:29,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:50:29,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:50:29,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:31,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:31,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:32,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:34,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:35,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:37,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:37,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:37,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:39,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:42,752 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:50:45,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:50:45,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:50:49,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:50,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:50:53,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:50:59,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:00,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:51:00,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:51:01,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:02,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:02,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:02,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:02,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:03,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:07,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:10,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:12,827 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:51:14,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:51:15,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:51:18,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:20,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:20,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:20,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:24,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:24,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:26,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:28,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:30,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:51:30,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:51:32,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:33,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:35,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:36,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:37,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:38,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:40,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:40,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:41,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:41,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:42,901 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:51:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:43,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:44,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:46,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:51:46,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:51:46,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:48,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:51,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:52,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:51:54,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:56,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:51:58,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:00,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:00,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:52:01,686 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:52:02,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:02,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:03,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:04,012 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 17:52:04,203 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 17:52:04,305 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 17:52:04,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 17:52:04,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:05,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:06,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:06,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:06,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:08,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:10,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:12,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:13,027 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:52:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:14,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:15,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:16,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:16,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:52:16,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:52:18,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:20,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:20,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:21,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:21,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:22,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:23,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:23,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:24,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:24,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:26,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:28,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:28,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:29,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:30,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:31,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:31,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:52:32,227 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:52:32,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:32,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:34,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:35,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:35,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:36,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:37,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:38,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:40,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:43,099 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:52:47,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:52:47,516 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:52:48,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:50,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:51,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:52,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:56,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:52:59,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:52:59,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:00,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:03,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:53:03,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:53:04,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:07,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:08,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:11,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:13,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:53:15,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:18,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:53:18,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:53:19,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:21,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:23,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:25,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:26,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:27,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:29,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:30,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:33,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:33,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:53:33,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:53:35,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:35,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:37,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:38,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:38,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:41,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:43,240 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:53:43,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:44,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:45,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:46,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:46,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:47,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:49,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:53:49,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:53:49,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:50,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:50,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:51,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:53,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:54,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:54,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:55,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:57,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:53:58,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:58,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:53:59,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:00,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:00,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:01,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:02,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:03,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:04,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:54:04,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:54:05,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:07,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:09,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:09,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:10,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:11,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:13,314 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:54:13,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:15,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:17,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:17,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:17,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:18,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:19,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:54:19,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:54:20,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:20,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:21,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:22,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:23,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:24,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:24,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:25,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:25,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:27,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:29,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:31,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:34,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:34,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:54:35,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:54:39,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:39,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:41,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:43,382 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:54:45,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:45,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:45,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:46,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:49,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:54:50,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:54:52,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:52,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:54:53,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:54:57,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:01,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:03,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:05,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:06,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:55:06,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:55:07,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:11,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:13,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:13,450 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:55:14,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:18,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:20,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:21,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:55:21,557 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:55:24,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:24,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:26,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:28,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:29,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:29,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:30,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:32,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:34,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:36,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:36,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:55:36,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:55:37,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:38,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:40,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:40,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:43,520 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:55:44,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:44,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:45,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:46,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:48,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:50,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:51,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:55:52,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:55:52,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:52,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:53,291 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 17:55:53,365 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 17:55:53,419 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 17:55:53,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 17:55:54,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:56,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:56,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:58,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:55:59,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:55:59,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:00,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:01,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:02,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:04,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:06,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:08,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:56:08,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:56:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:08,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:09,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:10,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:12,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:13,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:56:14,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:14,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:15,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:16,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:17,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:17,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:18,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:19,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:19,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:20,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:21,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:21,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:22,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:23,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:56:23,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:56:24,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:24,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:25,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:25,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:26,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:28,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:35,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:38,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:38,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:56:38,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:56:43,684 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:56:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:44,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:45,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:45,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:48,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:56:52,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:56:54,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:56:56,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:56:58,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:02,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:04,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:04,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:06,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:07,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:08,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:09,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:57:09,528 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:57:12,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:13,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:13,753 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:57:14,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:15,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:18,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:20,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:22,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:57:24,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:57:26,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:28,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:29,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:30,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:30,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:32,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:33,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:36,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:38,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:38,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:39,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:39,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:57:40,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:57:40,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:42,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:43,827 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:57:44,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:44,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:45,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:46,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:47,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:47,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:48,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:49,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:49,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:49,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:49,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:49,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:50,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:51,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:52,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:53,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:55,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:57,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:57:57,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:57:57,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:57:58,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:58,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:57:59,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:01,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:01,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:01,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:03,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:05,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:07,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:09,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:10,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:10,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:11,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:13,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:58:13,227 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:58:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:13,886 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:58:13,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:15,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:17,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:17,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:19,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:19,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:19,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:20,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:22,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:23,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:25,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:28,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:58:28,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:58:28,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:28,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:33,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:34,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:36,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:39,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:41,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:43,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:43,957 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:58:44,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:58:44,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:58:46,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:47,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:49,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:49,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:51,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:53,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:53,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:58:55,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:58:59,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:58:59,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:58:59,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:01,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:05,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:08,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:08,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:09,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:11,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:12,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:12,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:14,029 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:59:15,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:59:15,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:59:15,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:15,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:15,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:16,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:17,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:19,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:20,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:23,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:25,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:27,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:28,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:28,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:30,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:59:30,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:59:31,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:31,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:32,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:32,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:33,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:35,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:37,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:38,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:38,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:39,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:40,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:41,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:42,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:43,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:44,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 17:59:45,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:45,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 17:59:46,204 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 17:59:46,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 17:59:46,960 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 17:59:47,186 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 17:59:47,249 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 17:59:48,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 17:59:48,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:49,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:50,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:52,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:54,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:56,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 17:59:58,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:00,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:01,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:00:01,567 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:00:02,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:04,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:07,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:09,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:11,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:13,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:14,219 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:00:15,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:16,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:00:16,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:00:17,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:19,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:21,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:23,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:25,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:27,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:29,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:31,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:32,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:00:32,404 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:00:33,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:35,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:37,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:39,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:41,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:43,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:44,299 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:00:45,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:47,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:00:47,773 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:00:49,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:51,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:53,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:55,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:57,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:00:59,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:01,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:03,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:01:03,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:01:05,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:07,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:09,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:11,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:13,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:14,374 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:01:15,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:17,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:19,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:19,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:01:19,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:01:21,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:23,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:25,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:27,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:29,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:32,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:34,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:34,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:01:34,554 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:01:36,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:38,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:40,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:42,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:44,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:44,448 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:01:46,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:48,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:49,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:01:49,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:01:50,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:52,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:54,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:56,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:01:58,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:00,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:02,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:04,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:05,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:02:05,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:02:06,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:08,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:10,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:12,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:14,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:14,523 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:02:16,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:18,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:20,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:20,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:02:20,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:02:22,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:24,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:26,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:28,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:30,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:32,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:34,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:36,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:36,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:02:36,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:02:38,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:40,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:42,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:44,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:44,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:02:46,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:48,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:50,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:52,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:02:52,003 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:02:52,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:54,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:56,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:02:58,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:00,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:02,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:04,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:06,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:07,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:03:07,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:03:08,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:10,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:12,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:14,676 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:03:15,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:17,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:19,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:21,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:23,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:23,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:03:23,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:03:25,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:27,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:29,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:31,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:33,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:35,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:37,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:39,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:39,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:03:39,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:03:41,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:43,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:44,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:03:45,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:47,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:49,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:51,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:53,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:54,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:03:54,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:03:55,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:57,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:03:59,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:01,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:03,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:05,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:07,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:09,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:10,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:04:10,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:04:11,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:13,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:14,835 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:04:15,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:17,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:19,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:21,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:23,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:25,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:25,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:04:25,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:04:28,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:30,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:32,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:34,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:36,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:38,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:40,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:41,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:04:41,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:04:42,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:44,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:44,910 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:04:46,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:48,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:50,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:52,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:54,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:56,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:04:56,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:04:56,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:04:58,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:00,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:02,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:04,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:06,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:08,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:10,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:11,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:05:11,691 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:05:12,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:14,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:14,977 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:05:16,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:18,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:20,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:22,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:24,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:26,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:27,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:05:27,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:05:28,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:30,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:32,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:34,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:36,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:38,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:40,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:42,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:05:42,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:05:44,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:45,052 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:05:46,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:48,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:50,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:52,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:54,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:56,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:58,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:05:58,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:05:58,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:06:00,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:02,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:05,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:07,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:09,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:11,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:13,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:13,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:06:13,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:06:15,129 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:06:15,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:17,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:19,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:21,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:23,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:25,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:27,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:29,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:06:29,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:06:29,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:31,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:33,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:35,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:37,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:39,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:41,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:43,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:44,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:06:44,547 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:06:45,204 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:06:45,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:47,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:49,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:51,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:53,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:55,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:57,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:59,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:06:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:06:59,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:07:01,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:03,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:05,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:07,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:09,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:11,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:13,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:15,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:07:15,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:07:15,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:17,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:19,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:20,275 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:07:21,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:23,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:25,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:27,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:29,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:31,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:33,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:35,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:35,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:07:35,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:07:37,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:39,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:41,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:44,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:46,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:48,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:50,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:52,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:54,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:55,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:07:56,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:07:58,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:00,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:02,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:04,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:06,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:08,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:09,036 ERROR SenderThread:384207 [retry.py:__call__():126] Retry attempt failed: Traceback (most recent call last): File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.9/http/client.py", line 1371, in getresponse response.begin() File "/usr/lib/python3.9/http/client.py", line 319, in begin version, status, reason = self._read_status() File "/usr/lib/python3.9/http/client.py", line 280, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/usr/lib/python3.9/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/requests/adapters.py", line 440, in send resp = conn.urlopen( File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 785, in urlopen retries = retries.increment( File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment raise six.reraise(type(error), error, _stacktrace) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/packages/six.py", line 770, in reraise raise value File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 451, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.wandb.ai', port=443): Read timed out. (read timeout=10) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/wandb/sdk/lib/retry.py", line 102, in __call__ result = self._call_fn(*args, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/wandb/sdk/internal/internal_api.py", line 140, in execute return self.client.execute(*args, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/client.py", line 52, in execute result = self._get_result(document, *args, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/client.py", line 60, in _get_result return self.transport.execute(document, *args, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/transport/requests.py", line 38, in execute request = requests.post(self.url, **post_args) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/requests/api.py", line 117, in post return request('post', url, data=data, json=json, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/requests/sessions.py", line 529, in request resp = self.send(prep, **send_kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/requests/sessions.py", line 645, in send r = adapter.send(request, **kwargs) File "/mnt/lv_ai_1_dante/ml/rolvb/venv/lib/python3.9/site-packages/requests/adapters.py", line 532, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.wandb.ai', port=443): Read timed out. (read timeout=10) 2022-06-23 18:08:10,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:12,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:14,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:15,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:08:16,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:18,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:20,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:22,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:24,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:26,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:28,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:30,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:32,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:34,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:35,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:08:36,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:38,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:40,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:42,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:44,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:46,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:48,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:50,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:52,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:54,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:55,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:08:56,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:08:58,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:00,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:02,268 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:09:02,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:02,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:03,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:04,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:05,192 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:09:05,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:06,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:09:06,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:07,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:09,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:11,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:13,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:15,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:15,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:09:15,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:15,798 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:09:17,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:19,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:21,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:23,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:25,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:27,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:29,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:30,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:09:30,800 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:31,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:33,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:35,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:37,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:39,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:41,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:43,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:45,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:45,657 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:09:46,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:09:46,516 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:09:47,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:49,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:51,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:53,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:55,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:57,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:09:59,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:01,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:10:02,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:10:03,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:05,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:07,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:09,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:11,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:13,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:15,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:15,731 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:10:17,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:17,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:10:17,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:10:19,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:21,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:23,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:25,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:27,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:29,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:31,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:33,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:10:33,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:10:35,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:37,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:40,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:42,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:44,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:45,805 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:10:46,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:48,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:48,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:10:48,687 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:10:50,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:52,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:54,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:56,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:10:58,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:00,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:02,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:04,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:11:04,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:11:04,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:06,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:08,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:10,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:12,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:14,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:15,881 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:11:16,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:18,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:19,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:11:19,740 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:11:20,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:22,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:24,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:26,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:28,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:30,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:32,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:34,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:35,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:11:35,430 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:11:36,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:38,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:40,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:42,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:44,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:45,961 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:11:46,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:48,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:50,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:50,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:11:50,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:11:52,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:54,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:56,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:11:58,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:00,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:03,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:05,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:06,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:12:06,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:12:07,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:09,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:11,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:13,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:15,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:16,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:12:17,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:19,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:21,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:21,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:12:21,685 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:12:23,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:25,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:27,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:29,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:31,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:33,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:35,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:36,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:12:36,997 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:12:37,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:39,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:41,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:43,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:45,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:46,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:12:47,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:49,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:51,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:52,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:12:52,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:12:53,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:55,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:57,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:12:59,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:01,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:03,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:05,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:07,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:08,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:13:08,122 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:13:09,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:11,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:13,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:15,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:16,185 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:13:17,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:19,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:21,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:23,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:13:23,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:13:24,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:26,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:28,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:30,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:32,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:34,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:36,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:38,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:39,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:13:39,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:13:40,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:42,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:44,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:46,261 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:13:46,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:48,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:50,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:52,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:54,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:55,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:13:55,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:13:56,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:13:58,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:00,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:02,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:04,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:06,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:08,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:10,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:10,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:14:10,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:14:12,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:14,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:16,341 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:14:16,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:18,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:20,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:22,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:24,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:25,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:14:25,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:14:26,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:28,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:30,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:32,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:34,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:36,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:38,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:40,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:41,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:14:41,175 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:14:42,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:44,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:46,416 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:14:46,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:48,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:50,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:52,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:54,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:56,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:14:56,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:14:56,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:14:58,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:00,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:02,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:04,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:06,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:08,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:10,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:11,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:15:11,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:15:12,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:14,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:16,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:16,492 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:15:18,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:20,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:22,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:25,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:27,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:27,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:15:27,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:15:29,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:31,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:33,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:35,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:37,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:39,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:41,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:43,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:15:43,056 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:15:43,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:45,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:46,569 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:15:47,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:49,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:51,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:53,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:55,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:57,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:15:58,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:15:58,355 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:15:59,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:01,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:03,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:05,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:07,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:09,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:11,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:13,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:13,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:16:13,651 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:16:15,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:16,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:16:17,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:19,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:21,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:23,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:25,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:27,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:29,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:16:29,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:16:29,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:31,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:33,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:35,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:37,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:39,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:41,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:43,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:16:44,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:16:45,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:46,722 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:16:47,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:49,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:51,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:53,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:55,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:57,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:59,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:16:59,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:16:59,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:17:01,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:03,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:05,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:08,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:10,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:12,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:14,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:15,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:17:15,282 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:17:16,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:16,796 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:17:18,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:20,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:22,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:24,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:26,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:28,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:30,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:30,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:17:30,561 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:17:32,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:34,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:36,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:38,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:40,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:42,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:44,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:17:46,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:17:46,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:46,872 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:17:48,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:50,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:52,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:54,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:56,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:17:58,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:00,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:01,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:18:01,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:18:02,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:04,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:06,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:08,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:10,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:12,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:14,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:16,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:16,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:18:16,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:18:16,948 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:18:18,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:20,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:22,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:24,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:26,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:29,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:31,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:32,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:18:32,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:18:33,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:35,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:37,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:39,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:41,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:43,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:45,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:47,021 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:18:47,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:18:47,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:18:47,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:49,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:51,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:53,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:55,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:57,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:18:59,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:01,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:02,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:19:02,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:19:03,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:05,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:07,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:09,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:11,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:13,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:15,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:17,095 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:19:17,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:18,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:19:18,147 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:19:19,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:21,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:23,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:26,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:28,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:30,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:32,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:33,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:19:33,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:19:34,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:36,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:38,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:40,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:42,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:44,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:46,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:47,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:19:48,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:48,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:19:48,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:19:50,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:52,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:54,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:56,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:19:58,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:00,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:02,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:04,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:20:05,037 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:20:06,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:08,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:10,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:12,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:14,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:16,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:17,246 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:20:18,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:20,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:20,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:20:20,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:20:22,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:24,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:26,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:28,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:30,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:32,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:34,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:36,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:20:36,336 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:20:36,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:38,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:40,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:42,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:44,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:46,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:47,320 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:20:48,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:50,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:51,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:20:51,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:20:52,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:54,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:56,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:20:58,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:00,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:03,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:05,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:07,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:08,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:21:08,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:21:09,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:11,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:13,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:15,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:17,394 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:21:17,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:19,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:21,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:23,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:24,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:21:24,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:21:25,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:27,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:29,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:31,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:33,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:35,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:37,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:39,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:21:39,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:21:39,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:41,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:43,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:45,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:47,469 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:21:47,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:49,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:51,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:53,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:55,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:21:55,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:21:55,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:57,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:21:59,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:01,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:03,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:05,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:07,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:09,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:10,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:22:10,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:22:11,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:13,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:15,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:17,539 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:22:17,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:19,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:22,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:24,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:25,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:22:25,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:22:26,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:28,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:30,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:32,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:34,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:36,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:38,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:40,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:41,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:22:41,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:22:42,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:44,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:46,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:47,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:22:48,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:50,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:52,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:54,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:56,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:22:56,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:22:56,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:22:58,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:00,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:02,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:04,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:06,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:08,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:10,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:23:11,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:23:12,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:14,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:16,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:17,689 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:23:18,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:20,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:22,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:24,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:26,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:27,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:23:27,233 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:23:28,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:30,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:32,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:34,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:36,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:38,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:40,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:23:42,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:23:42,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:44,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:46,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:47,765 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:23:48,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:50,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:52,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:54,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:56,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:23:57,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:23:57,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:23:58,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:00,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:02,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:04,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:06,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:08,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:11,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:13,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:24:13,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:24:13,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:15,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:17,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:17,836 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:24:19,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:21,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:23,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:25,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:27,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:24:28,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:24:29,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:31,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:33,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:35,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:37,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:39,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:41,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:43,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:24:43,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:24:43,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:45,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:47,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:47,911 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:24:49,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:51,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:53,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:55,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:57,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:24:58,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:24:58,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:24:59,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:01,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:03,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:05,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:07,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:09,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:11,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:13,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:14,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:25:14,344 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:25:15,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:17,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:17,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:25:19,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:21,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:23,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:25,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:27,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:29,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:25:29,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:25:29,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:31,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:33,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:35,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:37,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:39,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:41,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:43,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:44,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:25:44,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:25:45,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:47,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:48,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:25:49,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:51,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:53,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:55,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:57,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:25:59,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:00,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:26:00,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:26:01,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:04,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:06,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:08,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:10,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:12,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:14,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:15,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:26:15,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:26:16,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:18,123 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:26:18,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:20,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:22,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:24,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:26,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:28,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:30,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:31,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:26:31,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:26:32,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:34,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:36,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:38,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:40,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:42,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:44,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:46,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:26:46,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:26:46,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:48,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:26:48,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:50,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:52,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:54,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:56,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:26:58,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:00,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:01,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:27:01,827 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:27:02,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:04,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:06,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:08,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:10,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:12,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:14,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:16,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:17,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:27:17,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:27:18,266 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:27:19,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:21,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:23,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:25,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:27,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:29,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:31,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:32,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:27:32,649 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:27:33,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:35,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:37,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:39,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:41,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:43,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:45,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:47,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:47,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:27:47,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:27:48,343 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:27:49,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:51,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:53,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:55,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:57,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:27:59,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:01,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:03,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:28:03,198 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:28:03,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:05,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:07,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:09,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:11,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:13,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:15,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:17,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:18,415 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:28:18,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:28:18,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:28:19,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:21,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:23,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:25,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:27,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:29,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:31,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:33,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:28:33,795 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:28:33,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:35,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:37,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:39,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:41,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:43,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:45,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:47,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:48,484 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:28:49,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:28:49,238 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:28:50,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:52,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:54,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:56,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:28:58,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:00,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:02,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:04,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:29:05,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:29:06,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:08,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:10,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:12,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:14,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:16,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:18,561 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:29:18,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:20,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:20,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:29:20,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:29:22,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:24,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:26,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:28,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:30,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:32,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:34,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:36,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:29:36,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:29:36,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:38,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:40,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:42,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:44,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:46,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:48,627 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:29:48,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:50,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:51,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:29:51,813 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:29:52,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:54,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:56,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:29:58,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:00,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:02,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:04,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:06,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:07,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:30:07,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:30:08,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:10,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:12,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:14,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:16,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:18,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:30:19,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:21,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:22,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:30:22,500 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:30:23,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:25,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:27,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:29,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:31,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:33,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:35,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:37,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:37,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:30:37,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:30:39,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:41,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:43,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:45,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:47,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:48,755 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:30:49,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:51,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:53,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:54,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:30:54,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:30:55,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:57,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:30:59,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:01,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:03,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:05,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:07,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:09,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:10,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:31:10,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:31:11,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:13,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:15,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:17,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:18,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:31:19,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:21,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:23,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:25,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:31:25,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:31:25,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:27,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:29,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:31,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:33,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:35,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:37,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:40,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:41,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:31:41,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:31:42,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:44,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:46,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:48,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:48,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:31:50,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:52,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:54,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:56,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:31:56,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:31:56,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:31:58,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:00,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:02,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:04,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:06,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:08,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:10,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:11,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:32:11,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:32:12,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:14,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:16,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:18,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:18,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:32:20,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:22,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:24,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:26,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:27,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:32:27,061 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:32:28,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:30,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:32,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:34,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:36,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:38,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:40,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:32:42,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:32:42,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:44,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:46,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:48,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:49,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:32:50,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:52,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:54,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:56,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:32:57,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:32:57,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:32:58,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:00,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:02,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:04,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:06,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:08,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:10,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:12,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:33:12,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:33:13,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:15,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:17,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:19,107 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:33:19,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:21,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:23,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:25,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:27,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:28,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:33:28,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:33:29,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:31,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:33,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:35,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:37,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:39,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:41,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:43,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:33:43,544 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:33:43,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:45,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:47,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:49,183 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:33:49,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:51,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:53,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:55,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:57,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:33:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:33:58,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:33:59,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:01,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:03,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:05,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:07,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:09,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:11,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:13,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:34:14,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:34:15,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:17,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:19,256 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:34:19,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:21,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:23,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:25,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:27,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:34:29,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:34:29,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:31,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:33,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:35,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:38,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:40,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:42,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:34:44,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:34:44,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:46,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:48,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:49,334 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:34:50,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:52,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:54,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:56,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:34:58,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:00,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:35:00,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:35:00,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:02,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:04,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:06,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:08,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:10,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:12,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:14,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:15,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:35:15,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:35:16,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:18,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:19,408 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:35:20,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:22,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:24,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:26,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:28,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:35:30,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:35:30,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:32,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:34,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:36,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:38,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:40,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:42,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:44,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:46,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:35:46,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:35:46,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:48,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:49,485 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:35:50,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:52,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:54,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:56,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:35:58,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:00,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:01,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:36:01,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:36:02,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:04,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:06,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:08,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:10,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:12,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:14,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:36:16,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:36:16,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:18,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:19,561 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:36:20,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:22,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:24,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:27,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:29,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:31,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:32,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:36:32,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:36:33,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:35,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:37,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:39,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:41,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:43,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:46,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:36:47,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:36:48,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:49,638 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:36:50,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:52,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:54,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:56,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:36:58,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:00,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:02,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:37:02,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:37:04,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:06,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:08,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:10,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:12,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:14,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:16,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:18,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:18,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:37:18,026 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:37:19,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:37:20,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:22,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:24,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:26,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:28,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:30,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:32,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:37:33,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:37:34,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:36,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:38,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:40,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:42,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:44,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:46,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:48,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:48,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:37:48,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:37:49,790 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:37:50,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:52,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:54,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:56,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:37:58,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:00,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:02,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:03,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:38:03,995 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:38:04,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:06,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:08,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:10,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:12,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:14,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:16,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:18,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:19,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:38:19,227 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:38:19,867 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:38:20,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:23,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:25,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:27,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:29,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:31,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:33,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:34,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:38:34,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:38:35,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:37,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:39,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:41,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:43,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:45,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:47,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:49,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:49,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:38:49,826 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:38:50,076 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:38:51,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:53,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:55,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:57,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:38:59,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:01,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:03,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:05,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:05,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:39:05,121 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:39:07,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:09,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:11,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:13,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:15,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:17,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:19,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:20,016 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:39:20,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:39:20,416 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:39:21,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:23,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:25,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:28,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:30,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:32,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:34,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:35,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:39:35,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:39:36,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:38,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:40,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:42,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:44,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:46,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:48,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:50,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:50,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:39:50,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:39:50,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:39:52,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:54,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:56,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:39:58,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:00,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:02,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:04,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:06,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:40:06,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:40:08,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:10,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:12,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:14,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:16,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:18,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:20,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:20,167 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:40:21,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:40:21,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:40:22,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:24,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:26,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:28,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:30,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:32,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:34,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:36,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:36,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:40:36,951 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:40:38,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:40,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:42,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:44,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:46,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:48,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:50,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:50,242 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:40:52,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:40:52,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:40:54,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:56,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:40:58,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:00,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:02,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:04,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:07,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:41:07,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:41:09,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:11,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:13,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:15,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:17,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:19,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:20,317 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:41:21,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:22,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:41:22,827 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:41:23,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:25,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:27,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:29,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:31,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:33,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:35,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:37,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:41:38,080 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:41:39,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:41,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:41:50,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:41:53,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:41:53,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:42:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:42:08,595 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:42:20,468 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:42:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:42:23,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:42:39,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:42:39,164 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 18:42:39,216 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 18:42:39,289 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 18:42:39,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:42:39,369 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:42:40,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 18:42:40,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:42:41,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:42:45,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:42:50,549 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:42:54,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:42:54,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:43:03,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:43:09,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:43:09,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:43:20,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:43:25,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:43:25,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:43:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:43:40,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:43:50,715 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:43:55,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:43:55,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:44:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:44:10,941 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:44:20,799 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:44:26,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:44:26,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:44:41,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:44:41,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:44:50,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:44:56,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:44:56,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:45:12,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:45:12,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:45:20,967 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:45:27,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:45:27,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:45:42,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:45:42,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:45:51,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:45:57,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:45:57,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:46:13,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:46:13,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:46:21,119 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:46:30,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:46:30,040 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:46:45,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:46:45,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:46:51,194 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:47:00,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:47:00,508 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:47:01,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:01,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:02,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:03,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:04,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:05,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:07,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:07,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:08,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:09,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:11,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:12,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:12,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:13,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:13,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:15,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:15,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:47:15,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:47:16,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:16,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:17,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:19,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:21,263 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:47:21,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:22,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:22,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:23,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:24,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:24,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:25,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:26,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:26,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:27,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:27,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:29,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:30,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:30,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:47:31,020 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:47:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:31,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:34,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:35,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:35,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:39,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:42,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:46,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:47:46,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:47:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:48,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:49,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:49,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,339 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:47:51,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:51,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:52,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:52,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:56,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:47:58,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:47:58,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:01,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:48:01,506 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:48:02,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:05,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:06,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:08,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:08,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:10,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:10,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:12,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:14,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:16,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:48:16,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:48:18,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:20,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:21,397 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:48:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:23,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:24,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:24,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:26,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:28,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:31,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:48:32,109 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:48:32,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:34,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:36,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:38,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:40,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:40,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:40,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:41,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:44,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:45,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:46,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:48:47,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:48:47,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:48,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:50,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:51,471 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:48:52,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:52,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:54,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:56,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:56,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:57,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:57,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:58,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:48:59,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:48:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:00,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:02,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:02,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:49:02,830 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:49:04,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:04,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:04,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:06,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:07,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:07,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:08,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:08,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:10,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:10,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:12,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:14,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:16,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:16,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:17,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:18,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:49:18,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:49:18,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:18,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:19,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:21,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:21,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:21,541 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:49:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:22,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:23,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:25,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:27,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:33,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:49:33,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:49:35,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:39,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:39,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:43,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:43,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:43,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:47,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:47,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:48,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:49:48,819 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:49:49,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:51,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:51,620 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:49:53,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:53,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:55,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:57,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:57,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:49:59,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:49:59,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:01,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:01,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:04,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:50:04,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:50:05,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:06,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:07,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:09,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:11,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:15,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:17,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:19,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:50:19,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:50:20,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:20,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:21,694 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:50:23,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:25,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:27,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:27,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:29,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:33,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:50:34,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:50:35,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:37,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:39,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:39,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:41,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:43,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:45,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:45,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:45,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:46,413 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 18:50:46,623 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 18:50:46,702 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 18:50:47,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 18:50:47,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:47,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:47,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:49,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:49,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:49,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:49,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:49,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:49,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:50:50,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:50:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:50,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:51,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:50:51,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:53,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:53,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:55,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:55,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:56,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:57,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:50:58,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:50:59,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:01,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:02,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:03,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:04,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:04,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:05,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:05,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:51:05,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:51:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:06,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:07,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:09,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:11,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:12,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:12,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:13,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:15,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:17,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:18,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:18,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:19,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:20,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:51:20,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:51:21,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:21,892 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:51:29,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:29,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:29,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:34,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:35,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:36,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:51:36,157 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:51:36,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:36,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:39,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:44,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:48,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:48,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:50,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:51,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:51:51,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:51:51,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:51,951 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:51:52,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:52,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:54,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:56,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:56,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:58,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:51:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:51:59,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:00,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:04,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:04,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:06,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:06,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:52:06,688 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:52:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:07,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:08,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:09,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:10,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:12,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:14,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:17,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:18,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:20,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:21,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:21,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:52:21,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:52:22,237 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:52:22,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:24,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:24,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:25,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:26,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:30,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:32,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:34,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:36,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:52:37,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:52:38,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:39,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:39,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:39,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:39,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:40,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:41,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:42,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:44,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:45,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:45,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:47,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:48,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:48,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:49,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:50,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:51,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:51,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:52,081 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:52:52,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:52,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:52:52,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:52:53,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:53,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:54,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:55,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:56,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:58,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:52:58,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:52:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:00,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:02,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:03,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:03,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:04,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:06,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:06,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:06,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:07,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:53:07,844 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:53:08,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:08,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:08,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:10,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:12,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:14,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:15,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:15,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:20,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:22,157 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:53:22,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:23,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:53:23,109 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:53:26,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:27,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:27,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:30,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:30,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:32,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:34,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:38,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:53:38,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:53:38,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:42,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:45,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:45,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:46,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:46,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:48,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:50,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:50,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:52,232 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:53:52,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:53,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:53:53,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:53:54,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:53:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:56,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:53:58,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:00,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:00,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:03,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:03,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:04,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:06,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:06,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:06,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:08,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:09,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:54:09,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:54:11,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:12,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:13,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:15,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:16,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:17,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:19,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:19,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:21,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:22,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:54:23,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:23,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:54:24,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:54:27,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:29,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:31,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:32,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:32,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:33,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:33,138 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 18:54:33,356 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 18:54:33,424 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 18:54:33,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 18:54:33,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:34,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:35,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:36,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:36,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:37,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:38,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:39,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:54:39,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:54:41,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:42,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:43,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:45,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:45,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:47,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:47,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:47,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:49,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:49,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:50,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:51,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:52,401 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:54:53,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:53,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:53,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:55,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:54:55,206 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:54:55,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:57,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:54:59,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:54:59,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:01,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:02,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:03,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:05,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:07,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:55:10,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:55:17,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:22,479 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:55:23,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:25,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:25,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:55:25,827 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:55:27,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:27,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:31,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:33,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:35,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:37,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:37,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:39,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:40,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:55:41,438 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:55:42,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:42,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:43,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:45,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:46,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:46,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:49,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:51,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:52,555 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:55:53,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:54,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:55,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:56,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:55:56,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:55:57,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:55:57,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:57,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:55:58,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:00,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:01,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:03,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:05,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:09,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:11,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:56:13,411 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:56:13,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:13,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:13,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:15,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:15,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:17,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:18,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:20,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:22,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:22,630 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:56:22,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:22,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:26,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:28,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:28,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:56:28,726 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:56:28,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:28,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:28,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:28,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:30,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:32,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:34,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:35,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:35,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:36,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:37,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:38,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:42,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:43,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:56:44,003 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:56:44,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:45,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:45,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:46,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:48,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:48,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:48,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:49,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:50,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:51,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:51,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:52,703 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:56:52,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:52,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:54,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:56,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:57,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:56:58,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:56:59,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:56:59,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:57:00,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:01,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:02,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:04,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:12,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:14,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:57:14,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:57:16,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:22,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:22,771 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:57:25,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:26,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:30,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:57:30,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:57:30,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:32,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:33,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:33,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:34,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:36,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:37,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:37,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:40,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:41,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:41,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:42,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:45,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:57:45,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:57:46,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:47,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:48,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:51,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:51,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:52,841 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:57:53,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:53,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:54,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:54,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:57:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:55,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:56,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:57,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:57:59,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:01,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:01,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:58:01,830 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:58:03,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:03,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:05,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:05,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:07,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:08,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:08,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:09,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:11,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:11,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:13,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:15,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:15,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:17,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:58:17,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:58:18,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:18,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:21,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:22,910 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:58:23,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:23,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:23,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:24,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:25,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:25,184 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 18:58:25,394 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 18:58:25,459 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 18:58:25,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 18:58:25,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:27,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:29,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:31,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:32,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:58:32,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:58:33,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:33,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:35,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:37,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:37,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:39,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:39,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:39,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:41,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:42,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:43,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:44,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:44,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:45,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:47,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:58:48,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:58:48,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:48,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:49,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:50,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:50,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:51,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:52,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,027 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:58:53,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:58:53,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:55,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:57,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:58:59,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:02,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:03,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:59:03,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:59:11,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:11,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:17,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:18,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:59:18,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:59:21,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:23,100 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:59:25,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:29,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:32,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:33,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:34,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:59:34,197 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:59:36,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:36,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:37,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:39,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:42,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:43,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:45,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:49,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:49,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 18:59:49,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 18:59:49,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:51,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:53,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 18:59:55,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:56,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 18:59:57,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 18:59:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:01,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:03,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:04,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:00:05,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:00:05,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:06,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:08,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:09,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:09,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:10,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:12,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:14,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:14,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:16,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:16,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:18,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:19,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:19,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:00:20,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:00:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:20,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:22,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:23,241 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:00:24,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:24,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:24,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:25,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:26,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:28,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:28,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:28,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:30,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:32,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:34,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:36,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:00:36,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:00:36,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:38,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:39,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:39,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:40,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:42,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:44,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:44,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:45,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:46,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:46,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:48,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:49,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:00:52,095 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:00:52,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:52,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:52,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:53,313 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:00:54,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:54,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:55,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:55,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:56,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:00:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:00:58,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:00,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:01:07,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:01:08,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:10,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:12,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:16,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:18,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:20,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:01:22,796 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:01:23,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:23,603 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:01:24,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:28,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:32,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:32,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:35,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:01:38,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:01:38,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:38,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:39,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:43,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:43,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:45,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:47,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:48,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:48,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:51,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:52,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:53,459 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:01:53,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:55,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:01:56,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:01:56,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:01:58,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:58,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:01:59,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:00,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:00,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:01,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:02,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:02,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:03,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:05,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:05,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:06,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:07,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:07,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:08,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:09,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:11,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:11,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:02:11,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:02:13,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:15,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:15,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:16,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:17,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:18,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:18,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:19,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:19,911 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 19:02:20,119 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 19:02:20,196 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 19:02:20,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 19:02:21,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:22,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:22,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:23,605 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:02:23,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:25,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:25,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:27,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:02:27,223 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:02:27,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:29,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:31,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:33,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:35,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:36,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:36,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:37,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:38,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:38,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:39,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:40,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:41,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:42,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:02:42,594 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:02:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:42,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:43,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:44,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:45,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:45,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:47,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:48,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:49,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:49,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:50,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:51,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:52,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:53,679 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:02:53,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:02:57,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:02:57,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:02:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:02:58,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:03,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:07,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:10,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:11,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:13,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:03:13,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:03:15,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:19,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:23,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:23,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:23,763 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:03:27,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:29,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:03:29,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:03:29,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:31,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:31,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:33,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:35,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:35,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:35,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:39,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:40,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:40,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:41,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:44,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:03:44,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:03:45,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:47,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:47,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:48,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:49,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:49,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:52,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:53,832 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:03:56,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:57,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:57,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:03:58,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:03:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:03:59,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:04:00,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:01,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:01,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:02,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:04,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:04,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:05,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:06,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:08,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:08,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:10,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:12,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:13,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:13,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:13,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:13,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:14,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:14,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:04:15,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:04:15,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:16,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:16,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:18,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:19,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:19,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:20,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:22,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:23,906 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:04:24,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:24,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:26,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:28,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:29,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:29,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:30,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:31,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:04:31,772 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:04:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:32,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:34,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:35,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:35,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:36,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:37,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:37,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:38,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:39,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:40,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:42,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:43,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:44,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:45,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:46,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:47,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:04:47,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:04:48,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:04:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:04:53,976 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:05:00,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:02,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:05:02,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:05:04,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:05,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:05,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:08,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:12,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:16,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:18,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:18,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:05:18,422 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:05:20,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:21,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:21,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:24,052 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:05:24,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:26,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:30,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:32,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:34,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:34,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:35,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:05:35,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:05:36,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:36,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:38,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:42,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:43,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:44,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:46,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:49,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:49,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:50,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:05:50,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:05:50,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:52,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:54,124 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:05:54,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:54,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:56,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:56,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:56,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:58,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:05:59,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:05:59,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:00,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:03,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:05,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:05,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:06:05,837 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:06:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:06,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:07,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:09,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:09,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:10,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:10,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:11,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:06:11,216 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 19:06:11,433 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 19:06:11,491 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 19:06:11,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 19:06:11,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:12,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:13,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:15,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:17,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:19,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:06:21,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:06:21,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:23,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:24,224 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:06:25,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:27,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:29,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:31,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:33,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:35,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:36,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:06:36,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:06:37,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:39,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:41,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:43,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:45,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:47,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:49,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:51,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:52,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:06:52,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:06:53,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:54,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:06:55,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:57,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:06:59,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:01,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:03,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:05,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:07,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:08,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:07:08,204 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:07:09,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:11,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:13,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:15,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:17,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:19,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:21,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:07:23,713 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:07:23,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:24,370 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:07:25,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:27,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:29,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:31,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:33,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:36,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:38,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:39,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:07:39,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:07:40,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:42,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:44,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:46,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:48,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:50,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:52,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:54,444 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:07:54,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:07:54,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:07:54,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:56,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:07:58,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:00,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:02,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:04,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:06,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:08,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:08:09,721 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:08:10,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:12,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:14,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:16,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:18,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:20,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:22,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:24,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:08:24,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:24,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:08:24,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:08:26,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:28,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:30,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:32,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:34,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:36,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:38,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:40,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:08:40,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:08:40,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:42,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:44,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:46,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:48,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:51,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:53,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:54,584 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:08:55,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:55,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:08:55,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:08:57,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:08:59,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:01,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:03,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:05,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:07,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:09,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:09:11,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:09:11,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:13,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:15,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:17,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:19,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:21,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:23,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:24,660 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:09:25,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:09:26,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:09:27,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:29,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:31,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:33,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:35,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:37,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:39,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:41,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:41,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:09:41,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:09:43,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:45,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:47,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:49,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:51,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:53,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:54,734 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:09:55,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:09:57,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:09:57,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:09:59,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:01,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:03,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:05,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:07,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:09,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:11,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:12,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:10:12,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:10:13,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:15,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:17,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:19,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:21,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:23,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:24,811 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:10:25,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:27,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:10:27,767 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:10:27,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:29,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:31,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:33,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:35,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:37,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:39,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:41,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:43,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:10:43,128 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:10:43,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:45,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:47,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:49,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:51,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:53,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:54,885 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:10:55,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:57,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:10:58,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:10:58,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:10:59,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:01,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:04,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:06,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:08,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:10,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:12,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:13,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:11:13,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:11:14,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:16,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:18,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:20,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:22,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:24,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:24,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:11:26,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:28,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:29,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:11:29,060 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:11:30,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:32,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:34,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:36,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:38,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:40,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:42,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:44,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:11:44,457 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:11:44,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:46,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:48,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:50,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:52,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:54,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:55,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:11:56,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:58,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:11:59,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:11:59,754 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:12:00,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:02,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:04,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:06,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:08,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:10,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:12,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:14,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:15,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:12:15,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:12:16,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:18,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:20,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:22,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:24,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:25,111 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:12:26,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:28,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:30,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:12:30,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:12:30,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:32,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:34,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:36,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:38,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:41,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:43,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:45,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:12:45,731 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:12:45,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:47,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:49,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:51,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:53,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:55,184 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:12:55,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:57,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:12:59,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:01,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:13:01,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:13:01,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:03,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:05,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:07,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:09,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:11,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:13,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:15,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:16,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:13:16,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:13:17,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:19,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:21,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:23,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:25,260 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:13:25,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:27,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:29,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:31,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:13:31,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:13:31,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:33,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:35,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:37,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:39,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:41,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:43,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:45,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:13:47,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:13:47,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:49,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:52,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:54,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:55,333 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:13:56,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:13:58,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:00,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:02,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:02,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:14:02,795 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:14:04,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:06,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:08,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:10,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:12,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:14,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:16,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:18,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:18,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:14:18,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:14:20,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:23,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:25,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:25,408 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:14:27,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:29,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:31,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:33,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:14:33,346 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:14:35,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:37,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:39,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:41,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:43,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:45,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:47,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:48,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:14:48,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:14:49,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:51,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:53,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:55,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:55,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:14:57,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:14:59,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:01,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:03,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:03,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:15:03,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:15:05,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:07,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:09,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:11,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:13,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:15,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:17,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:19,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:19,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:15:19,363 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:15:21,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:23,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:25,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:25,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:15:27,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:29,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:31,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:33,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:15:34,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:15:35,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:37,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:39,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:41,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:43,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:45,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:47,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:49,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:15:50,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:15:51,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:53,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:55,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:55,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:15:57,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:15:59,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:01,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:03,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:05,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:05,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:16:05,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:16:08,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:10,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:12,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:14,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:16,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:18,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:20,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:20,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:16:20,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:16:22,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:24,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:25,702 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:16:26,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:28,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:30,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:32,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:34,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:16:36,058 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:16:36,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:38,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:40,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:42,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:44,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:46,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:48,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:50,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:51,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:16:51,425 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:16:52,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:54,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:55,775 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:16:56,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:16:58,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:00,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:02,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:04,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:06,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:17:06,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:17:07,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:09,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:11,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:13,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:15,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:17,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:19,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:21,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:22,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:17:22,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:17:23,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:25,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:25,852 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:17:27,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:29,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:31,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:33,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:35,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:37,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:17:37,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:17:39,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:41,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:43,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:45,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:47,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:49,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:51,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:52,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:17:52,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:17:53,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:55,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:55,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:17:57,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:17:59,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:01,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:03,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:05,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:07,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:18:08,104 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:18:09,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:11,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:13,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:15,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:17,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:20,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:22,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:18:23,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:18:24,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:26,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:18:26,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:28,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:30,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:32,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:34,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:36,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:38,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:38,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:18:38,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:18:40,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:42,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:44,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:46,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:48,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:50,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:52,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:18:54,111 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:18:54,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:56,083 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:18:56,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:18:58,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:00,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:02,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:04,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:06,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:08,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:09,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:19:09,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:19:10,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:12,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:14,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:16,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:18,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:20,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:22,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:24,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:24,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:19:24,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:19:26,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:26,157 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:19:28,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:30,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:32,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:34,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:36,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:38,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:19:40,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:19:40,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:42,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:44,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:46,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:48,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:50,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:52,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:54,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:55,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:19:55,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:19:56,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:19:56,226 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:19:58,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:00,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:02,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:05,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:07,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:09,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:10,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:20:10,949 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:20:11,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:13,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:15,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:17,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:19,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:21,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:23,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:25,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:26,301 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:20:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:20:26,321 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:20:27,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:29,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:31,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:33,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:35,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:37,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:39,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:41,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:41,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:20:41,596 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:20:43,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:45,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:47,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:49,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:51,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:53,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:55,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:56,376 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:20:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:20:56,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:20:57,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:20:59,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:01,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:03,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:05,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:07,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:09,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:11,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:12,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:21:12,150 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:21:13,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:15,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:17,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:19,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:21,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:23,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:25,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:26,449 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:21:27,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:21:27,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:21:29,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:31,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:33,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:36,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:38,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:40,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:42,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:42,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:21:42,857 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:21:44,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:46,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:48,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:50,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:52,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:54,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:56,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:21:56,526 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:21:58,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:21:58,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:21:58,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:00,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:02,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:04,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:06,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:08,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:10,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:12,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:13,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:22:13,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:22:14,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:16,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:18,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:20,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:22,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:24,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:26,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:26,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:22:28,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:28,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:22:28,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:22:30,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:32,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:34,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:36,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:38,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:40,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:42,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:44,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:22:44,119 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:22:44,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:46,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:48,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:50,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:52,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:54,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:56,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:56,677 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:22:58,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:22:59,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:22:59,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:23:00,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:02,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:04,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:06,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:08,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:10,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:12,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:14,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:14,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:23:14,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:23:16,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:18,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:20,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:22,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:24,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:26,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:26,751 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:23:28,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:29,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:23:29,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:23:30,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:32,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:34,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:36,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:38,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:40,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:43,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:45,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:45,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:23:45,406 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:23:47,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:49,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:51,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:53,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:55,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:56,825 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:23:57,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:23:59,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:01,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:24:01,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:24:01,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:03,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:05,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:07,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:09,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:11,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:13,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:15,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:16,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:24:16,433 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:24:17,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:19,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:21,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:23,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:25,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:26,900 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:24:27,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:29,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:31,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:31,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:24:31,747 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:24:33,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:35,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:37,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:39,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:42,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:44,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:46,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:47,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:24:47,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:24:48,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:50,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:52,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:54,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:56,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:24:56,974 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:24:58,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:00,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:02,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:02,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:25:02,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:25:04,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:06,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:08,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:10,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:12,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:14,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:16,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:18,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:25:18,012 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:25:18,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:20,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:22,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:24,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:26,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:27,049 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:25:28,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:30,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:32,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:33,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:25:33,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:25:34,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:36,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:38,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:40,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:42,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:44,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:46,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:48,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:48,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:25:48,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:25:50,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:52,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:54,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:56,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:25:57,127 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:25:58,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:01,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:03,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:04,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:26:04,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:26:05,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:07,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:09,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:11,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:13,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:15,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:17,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:19,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:20,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:26:20,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:26:21,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:23,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:25,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:27,203 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:26:27,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:29,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:31,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:33,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:35,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:35,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:26:35,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:26:37,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:39,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:41,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:43,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:45,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:47,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:49,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:51,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:26:51,149 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:26:51,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:53,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:55,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:57,280 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:26:57,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:26:59,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:01,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:03,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:05,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:06,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:27:06,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:27:07,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:09,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:11,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:13,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:15,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:17,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:19,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:21,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:27:21,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:27:22,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:24,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:26,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:27,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:27:28,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:30,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:32,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:34,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:36,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:27:37,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:27:38,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:40,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:42,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:44,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:46,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:48,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:50,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:52,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:52,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:27:52,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:27:54,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:56,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:27:57,434 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:27:58,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:00,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:02,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:04,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:06,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:08,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:28:08,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:28:08,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:10,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:12,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:14,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:16,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:18,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:20,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:22,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:28:23,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:28:24,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:26,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:27,510 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:28:28,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:30,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:32,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:34,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:36,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:39,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:28:39,250 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:28:39,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:41,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:43,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:45,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:47,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:49,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:51,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:53,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:55,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:56,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:28:56,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:28:57,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:28:57,586 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:28:59,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:01,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:03,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:05,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:07,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:09,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:11,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:12,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:29:12,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:29:13,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:15,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:17,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:19,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:21,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:23,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:25,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:27,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:27,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:29:27,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:29:27,716 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:29:29,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:31,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:33,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:35,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:37,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:39,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:41,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:42,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:29:42,748 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:29:44,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:46,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:48,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:50,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:52,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:54,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:56,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:29:57,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:29:58,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:29:58,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:29:58,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:00,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:02,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:04,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:06,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:08,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:10,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:12,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:13,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:30:13,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:30:14,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:16,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:18,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:20,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:22,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:24,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:26,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:27,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:30:28,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:30:29,582 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:30:30,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:32,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:34,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:36,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:38,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:40,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:42,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:44,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:45,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:30:45,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:30:46,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:48,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:50,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:52,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:54,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:56,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:30:57,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:30:58,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:00,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:01,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:31:01,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:31:02,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:05,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:07,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:09,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:11,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:13,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:15,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:17,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:18,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:31:18,600 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:31:19,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:21,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:23,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:25,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:27,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:27,967 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:31:29,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:31,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:33,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:34,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:31:34,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:31:35,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:37,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:39,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:41,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:43,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:45,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:47,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:49,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:49,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:31:49,667 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:31:51,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:53,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:55,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:57,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:31:58,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:31:59,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:01,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:03,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:05,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:32:05,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:32:05,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:07,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:09,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:11,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:13,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:15,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:17,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:20,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:20,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:32:20,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:32:22,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:24,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:26,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:28,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:32:28,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:30,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:32,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:34,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:36,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:32:36,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:32:36,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:38,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:40,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:42,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:44,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:46,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:48,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:50,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:51,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:32:51,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:32:52,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:54,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:56,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:32:58,195 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:32:58,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:00,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:02,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:04,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:06,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:07,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:33:07,016 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:33:08,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:10,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:12,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:14,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:16,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:18,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:20,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:22,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:33:22,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:33:22,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:25,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:27,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:28,269 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:33:29,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:31,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:33,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:35,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:37,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:37,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:33:37,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:33:39,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:41,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:43,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:45,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:47,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:49,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:51,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:52,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:33:52,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:33:53,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:55,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:57,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:33:58,345 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:33:59,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:01,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:03,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:05,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:07,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:34:08,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:34:09,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:11,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:13,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:15,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:17,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:19,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:21,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:23,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:23,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:34:23,913 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:34:25,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:27,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:28,421 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:34:29,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:31,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:33,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:35,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:37,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:39,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:34:39,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:34:40,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:42,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:44,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:46,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:48,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:50,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:52,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:54,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:34:54,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:34:54,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:56,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:34:58,497 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:34:58,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:00,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:02,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:04,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:06,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:08,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:09,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:35:09,912 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:35:10,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:12,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:14,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:16,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:18,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:20,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:22,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:24,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:35:25,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:35:26,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:28,577 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:35:28,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:30,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:32,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:34,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:36,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:38,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:40,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:40,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:35:40,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:35:42,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:44,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:46,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:48,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:50,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:52,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:54,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:56,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:35:56,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:35:56,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:58,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:35:58,651 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:36:00,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:02,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:04,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:06,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:08,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:10,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:12,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:36:13,312 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:36:14,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:16,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:19,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:21,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:23,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:25,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:27,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:36:28,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:36:29,091 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:36:29,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:31,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:33,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:35,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:37,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:39,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:41,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:43,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:44,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:36:44,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:36:45,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:47,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:49,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:51,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:53,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:55,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:57,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:58,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:36:59,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:36:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:36:59,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:37:01,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:03,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:05,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:07,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:09,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:11,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:13,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:15,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:37:15,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:37:15,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:17,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:19,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:21,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:23,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:25,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:27,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:28,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:37:29,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:31,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:37:32,103 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:37:33,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:35,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:37,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:39,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:41,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:43,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:45,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:47,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:37:47,408 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:37:47,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:49,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:51,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:53,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:55,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:57,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:37:58,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:37:59,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:01,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:38:02,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:38:03,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:05,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:07,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:09,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:11,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:14,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:16,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:18,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:38:18,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:38:18,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:20,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:22,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:24,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:26,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:28,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:29,031 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:38:30,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:32,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:33,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:38:33,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:38:34,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:36,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:38,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:40,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:42,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:44,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:46,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:48,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:48,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:38:48,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:38:50,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:52,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:54,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:56,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:58,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:38:59,105 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:39:00,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:02,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:04,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:39:04,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:39:04,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:06,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:08,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:10,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:12,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:14,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:16,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:18,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:19,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:39:19,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:39:20,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:22,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:24,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:26,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:28,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:29,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:39:30,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:32,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:34,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:34,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:39:34,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:39:36,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:38,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:40,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:42,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:44,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:47,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:49,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:50,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:39:50,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:39:51,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:53,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:55,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:57,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:39:59,241 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:39:59,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:01,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:03,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:05,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:40:05,816 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:40:07,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:09,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:11,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:13,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:15,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:17,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:19,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:21,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:40:21,127 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:40:21,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:23,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:25,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:27,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:29,306 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:40:29,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:31,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:33,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:35,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:36,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:40:36,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:40:37,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:39,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:41,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:43,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:45,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:47,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:49,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:51,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:40:51,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:40:53,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:55,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:57,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:40:59,379 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:40:59,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:01,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:03,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:06,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:07,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:41:07,225 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:41:08,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:10,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:12,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:14,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:16,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:18,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:20,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:22,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:41:22,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:41:22,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:24,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:26,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:28,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:29,457 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:41:30,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:32,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:34,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:36,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:41:37,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:41:38,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:40,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:42,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:44,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:46,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:48,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:50,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:52,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:53,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:41:53,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:41:54,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:56,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:58,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:41:59,530 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:42:00,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:02,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:04,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:06,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:08,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:42:08,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:42:08,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:10,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:12,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:14,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:16,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:18,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:20,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:22,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:23,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:42:23,780 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:42:24,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:26,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:28,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:29,605 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:42:30,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:32,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:35,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:37,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:39,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:42:39,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:42:39,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:41,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:43,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:45,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:47,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:49,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:51,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:53,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:54,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:42:54,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:42:55,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:57,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:42:59,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:42:59,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:01,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:03,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:05,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:07,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:43:09,555 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:43:09,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:11,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:13,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:15,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:17,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:19,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:21,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:23,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:24,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:43:24,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:43:25,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:27,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:29,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:43:29,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:31,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:33,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:35,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:37,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:39,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:40,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:43:40,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:43:41,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:43,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:45,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:47,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:49,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:51,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:53,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:55,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:43:55,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:43:55,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:58,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:43:59,832 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:44:00,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:02,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:04,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:06,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:08,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:10,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:44:10,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:44:10,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:12,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:14,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:16,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:18,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:20,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:22,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:24,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:44:26,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:44:26,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:28,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:29,906 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:44:30,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:32,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:34,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:36,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:38,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:40,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:41,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:44:41,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:44:42,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:44,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:46,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:48,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:50,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:52,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:54,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:56,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:44:56,757 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:44:56,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:58,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:44:59,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:45:00,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:02,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:04,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:06,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:08,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:10,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:12,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:45:12,058 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:45:12,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:14,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:16,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:18,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:20,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:22,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:25,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:45:27,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:45:27,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:29,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:30,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:45:31,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:33,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:35,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:37,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:39,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:41,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:42,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:45:42,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:45:43,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:45,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:47,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:49,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:51,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:53,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:55,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:57,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:45:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:45:58,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:45:59,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:00,135 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:46:01,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:03,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:05,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:07,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:09,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:11,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:13,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:46:13,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:46:13,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:15,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:17,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:19,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:21,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:23,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:25,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:27,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:29,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:46:29,261 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:46:29,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:30,212 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:46:31,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:33,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:35,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:37,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:39,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:41,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:43,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:44,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:46:44,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:46:45,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:47,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:49,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:52,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:54,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:56,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:46:58,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:00,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:47:00,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:47:00,548 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:47:00,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:02,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:04,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:06,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:08,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:10,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:12,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:14,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:47:15,549 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:47:16,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:18,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:20,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:22,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:24,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:26,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:28,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:30,364 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:47:30,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:31,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:47:31,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:47:32,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:34,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:36,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:38,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:40,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:42,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:44,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:46,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:47:46,798 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:47:46,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:48,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:50,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:52,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:54,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:56,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:47:58,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:48:00,440 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:48:00,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:48:02,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:48:03,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:48:03,052 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:48:04,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:48:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:48:18,692 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:48:30,516 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:48:34,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:48:34,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:48:49,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:48:49,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:49:00,591 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:49:02,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:49:02,279 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 19:49:02,331 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 19:49:02,406 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 19:49:02,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 19:49:02,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:49:04,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:49:04,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:49:04,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:49:08,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:49:20,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:49:20,183 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:49:25,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:49:30,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:49:35,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:49:35,489 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:49:50,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:49:50,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:50:00,768 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:50:06,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:50:06,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:50:21,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:50:21,599 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:50:30,853 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:50:36,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:50:36,869 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:50:52,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:50:52,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:51:00,945 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:51:07,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:51:07,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:51:22,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:51:22,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:51:31,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:51:38,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:51:38,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:51:53,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:51:53,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:52:01,114 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:52:08,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:52:08,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:52:23,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:52:23,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:52:31,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:52:39,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:52:39,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:52:54,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:52:54,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:53:01,260 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:53:09,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:53:09,776 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:53:24,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:53:24,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:53:25,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:25,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:26,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:26,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:29,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:30,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:31,336 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:53:32,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:32,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:32,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:33,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:34,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:35,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:35,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:36,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:37,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:38,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:39,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:39,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:40,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:40,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:53:40,401 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:53:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:41,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:42,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:42,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:44,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:45,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:46,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:48,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:50,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:50,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:51,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:52,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:53,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:53,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:54,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:54,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:53:55,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:53:55,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:55,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:53:56,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:53:58,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:01,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:54:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:04,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:08,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:10,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:54:10,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:54:12,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:15,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:15,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:16,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:17,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:17,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:20,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:20,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:24,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:26,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:54:26,388 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:54:28,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:31,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:54:32,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:34,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:34,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:34,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:35,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:38,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:40,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:41,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:54:41,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:54:44,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:46,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:47,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:48,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:53,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:55,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:57,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:54:57,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:54:57,141 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:54:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:54:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:00,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:01,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:01,546 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:55:03,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:03,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:05,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:05,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:07,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:09,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:10,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:10,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:55:12,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:55:12,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:13,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:14,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:14,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:15,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:16,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:16,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:17,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:17,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:19,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:20,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:21,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:21,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:22,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:23,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:25,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:26,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:26,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:27,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:27,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:55:27,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:55:29,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:31,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:31,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:55:31,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:31,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:32,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:33,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:35,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:35,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:35,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:37,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:39,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:39,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:39,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:41,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:42,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:43,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:43,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:55:43,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:55:44,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:44,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:45,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:45,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:46,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:48,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:49,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:49,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:51,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:53,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:55:56,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:56,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:55:58,437 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:55:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:55:58,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:01,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:01,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:56:03,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:03,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:04,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:07,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:08,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:11,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:13,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:56:13,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:56:14,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:14,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:15,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:18,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:18,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:19,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:21,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:23,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:25,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:56:28,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:56:29,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:30,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:31,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:31,752 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:56:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:35,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:37,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:39,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:41,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:41,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:43,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:44,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:56:44,258 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:56:45,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:47,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:51,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:53,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:55,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:55,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:57,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:57,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:56:59,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:56:59,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:56:59,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:57:01,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:01,824 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:57:02,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:02,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:03,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:04,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:06,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:07,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:09,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:10,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:10,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:11,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:11,503 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 19:57:11,724 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 19:57:11,793 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 19:57:12,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 19:57:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:13,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:13,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:14,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:57:15,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:57:15,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:17,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:17,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:17,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:18,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:21,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:23,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:25,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:26,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:27,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:28,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:30,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:30,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:57:30,366 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:57:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:31,942 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:57:32,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:34,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:34,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:34,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:36,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:37,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:38,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:38,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:40,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:40,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:41,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:42,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:43,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:44,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:44,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:46,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:57:46,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:57:46,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:47,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:52,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:54,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:57:55,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:55,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:57:57,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:00,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:01,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:58:01,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:58:02,015 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:58:04,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:07,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:07,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:08,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:09,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:11,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:12,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:12,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:16,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:16,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:58:16,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:58:18,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:18,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:22,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:24,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:27,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:28,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:30,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:32,083 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:58:32,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:58:32,378 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:58:33,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:33,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:34,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:36,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:38,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:39,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:39,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:42,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:44,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:46,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:58:47,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:58:48,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:52,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:54,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:55,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:56,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:57,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:57,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:58,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:58:59,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:58:59,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:00,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:00,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:01,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:02,146 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:59:02,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:03,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:59:03,254 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:59:04,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:04,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:04,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:04,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:05,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:06,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:07,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:08,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:08,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:10,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:12,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:13,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:13,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:14,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:15,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:16,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:16,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:17,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:18,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:59:18,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:59:19,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:21,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:21,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:23,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:23,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:25,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:25,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:27,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:29,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:31,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:31,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:32,213 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 19:59:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:33,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:33,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:59:33,971 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:59:34,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:34,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:35,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:37,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:39,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:42,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:42,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:45,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 19:59:49,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 19:59:49,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:50,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:51,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:53,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:57,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 19:59:58,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 19:59:58,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,283 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:00:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:02,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:03,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:04,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:00:04,559 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:00:05,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:09,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:11,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:13,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:17,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:18,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:18,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:19,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:00:19,980 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:00:21,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:21,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:21,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:23,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:26,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:27,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:29,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:31,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:32,355 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:00:35,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:00:35,916 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:00:38,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:38,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:40,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:41,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:41,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:44,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:46,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:48,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:50,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:52,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:00:52,368 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:00:54,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:56,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:56,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:00:58,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:00:58,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:00,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:01,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:01,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,432 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:01:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:02,691 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 20:01:02,901 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 20:01:02,979 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 20:01:03,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:03,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 20:01:03,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:04,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:06,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:07,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:01:07,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:01:08,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:08,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:09,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:10,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:12,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:13,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:14,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:15,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:16,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:17,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:17,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:18,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:20,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:22,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:23,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:01:23,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:01:23,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:23,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:24,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:26,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:26,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:27,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:28,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:30,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:32,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:32,547 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:01:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:33,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:34,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:34,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:36,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:38,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:01:38,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:01:39,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:39,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:46,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:50,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:50,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:50,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:52,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:53,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:01:53,813 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:01:54,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:01:54,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:54,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:01:58,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:02,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:02,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:02:04,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:06,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:09,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:02:09,298 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:02:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:10,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:12,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:15,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:16,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:18,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:20,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:20,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:22,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:24,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:02:24,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:02:25,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:25,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:28,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:30,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:32,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:32,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:02:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:35,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:36,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:39,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:39,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:02:39,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:02:40,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:43,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:44,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:47,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:49,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:51,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:53,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:55,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:55,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:02:55,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:02:57,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:02:57,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:57,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:58,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:02:59,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:00,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:01,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:02,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,751 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:03:02,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:02,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:03,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:04,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:04,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:05,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:06,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:06,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:07,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:08,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:09,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:10,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:10,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:03:11,095 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:03:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:11,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:13,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:13,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:13,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:14,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:17,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:19,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:19,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:19,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:20,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:21,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:22,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:22,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:23,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:23,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:25,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:26,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:03:26,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:03:27,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:29,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:29,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:31,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:32,814 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:03:37,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:37,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:38,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:40,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:41,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:41,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:03:41,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:03:42,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:42,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:45,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:49,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:49,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:53,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:56,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:56,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:57,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:03:57,025 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:03:57,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:03:58,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:03:58,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:00,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:01,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:02,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:02,888 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:04:03,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:03,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:04,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:05,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:07,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:07,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:07,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:10,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:11,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:12,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:04:12,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:04:13,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:13,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:17,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:19,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:20,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:20,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:23,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:24,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:24,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:25,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:27,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:27,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:04:27,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:04:31,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:32,964 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:04:33,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:35,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:37,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:39,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:40,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:41,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:42,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:42,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:43,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:04:43,080 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:04:45,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:47,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:47,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:49,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:50,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:50,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:50,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:50,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:51,655 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 20:04:51,886 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 20:04:51,945 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 20:04:52,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 20:04:52,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:54,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:56,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:56,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:58,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:04:58,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:04:58,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:04:58,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:04:59,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:00,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:02,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:02,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:03,072 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:05:03,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:03,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:04,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:05,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:06,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:08,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:09,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:10,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:10,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:12,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:13,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:05:13,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:05:14,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:16,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:17,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:17,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:18,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:19,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:19,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:20,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:22,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:24,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:26,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:29,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:05:29,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:05:33,139 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:05:33,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:33,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:36,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:38,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:38,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:40,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:44,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:05:44,405 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:05:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:45,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:46,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:50,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:50,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:52,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:56,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:56,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:05:59,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:05:59,768 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:05:59,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:05:59,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:00,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:02,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:03,211 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:06:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:04,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:06,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:08,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:11,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:12,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:15,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:06:15,138 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:06:15,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:16,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:16,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:19,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:21,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:23,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:23,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:24,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:26,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:27,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:29,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:30,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:06:30,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:06:31,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:31,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:31,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:33,284 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:06:33,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:35,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:37,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:37,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:38,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:39,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:40,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:40,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:41,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:43,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:45,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:06:45,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:06:46,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:46,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:46,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:46,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:47,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:49,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:51,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:53,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:55,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:56,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:57,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:57,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:06:59,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:06:59,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:00,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:07:00,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:07:01,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:03,358 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:07:03,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:04,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:04,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:05,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:05,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:07,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:07,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:09,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:10,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:10,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:11,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:11,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:13,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:15,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:15,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:16,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:07:16,370 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:07:17,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:19,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:19,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:21,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:31,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:31,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:07:31,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:07:33,432 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:07:35,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:38,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:41,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:43,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:45,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:46,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:07:47,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:07:47,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:47,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:47,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:51,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:53,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:55,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:57,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:07:57,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:07:59,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:01,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:08:02,626 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:08:02,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:02,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:03,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:08:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:04,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:05,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:05,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:07,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:08,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:08,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:09,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:09,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:11,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:12,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:13,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:14,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:15,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:17,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:08:17,860 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:08:18,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:18,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:19,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:21,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:23,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:25,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:28,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:31,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:32,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:33,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:08:33,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:08:33,582 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:08:34,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:36,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:37,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:38,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:39,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:39,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:40,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:41,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:41,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:41,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:41,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:42,534 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 20:08:42,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:42,744 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 20:08:42,813 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 20:08:43,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 20:08:43,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:44,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:46,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:48,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:08:48,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:08:48,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:50,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:52,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:54,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:54,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:54,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:56,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:57,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:57,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:08:58,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:08:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:00,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:00,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:02,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:03,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:03,666 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:09:03,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:09:03,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:09:04,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:04,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:04,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:06,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:08,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:08,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:10,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:11,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:11,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:12,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:14,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:15,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:16,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:18,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:19,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:09:19,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:09:26,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:30,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:33,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:09:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:34,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:09:34,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:09:34,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:38,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:40,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:40,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:42,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:43,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:43,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:46,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:47,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:49,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:50,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:51,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:09:51,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:09:52,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:56,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:09:57,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:57,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:09:58,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:01,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:01,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:02,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:03,808 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:10:04,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:06,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:06,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:10:06,886 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:10:07,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:07,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:08,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:08,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:10,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:11,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:12,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:15,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:16,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:17,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:17,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:18,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:20,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:20,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:22,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:10:22,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:10:24,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:25,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:26,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:28,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:29,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:29,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:30,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:32,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:33,879 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:10:34,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:36,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:36,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:36,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:10:37,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:10:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:38,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:40,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:42,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:44,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:46,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:48,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:50,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:52,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:52,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:53,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:53,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:10:53,872 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:10:54,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:55,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:55,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:56,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:57,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:57,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:58,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:10:59,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:10:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:01,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:01,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:03,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:03,952 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:11:04,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:04,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:05,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:07,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:08,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:11:09,339 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:11:09,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:09,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:09,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:11,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:13,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:19,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:19,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:21,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:21,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:24,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:11:24,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:11:27,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:31,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:34,020 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:11:35,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:38,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:38,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:39,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:40,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:11:40,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:11:43,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:47,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:47,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:47,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:48,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:49,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:53,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:54,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:54,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:11:57,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:11:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:11:57,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:11:59,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:01,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:02,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:03,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:04,086 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:12:04,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:04,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:05,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:06,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:06,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:08,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:09,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:11,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:11,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:12,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:12,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:13,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:15,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:16,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:16,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:12:16,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:12:19,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:21,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:21,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:23,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:23,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:24,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:25,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:27,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:28,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:28,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:29,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:30,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:32,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:12:32,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:12:33,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:33,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:33,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:34,164 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:12:34,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:34,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:34,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:34,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:12:35,373 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 20:12:35,586 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 20:12:35,644 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 20:12:35,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 20:12:35,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:36,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:37,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:39,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:41,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:43,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:45,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:47,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:48,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:12:48,758 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:12:49,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:51,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:53,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:55,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:12:57,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:00,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:02,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:04,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:13:04,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:13:04,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:13:04,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:06,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:08,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:10,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:12,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:14,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:16,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:18,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:19,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:13:19,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:13:20,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:22,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:24,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:26,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:28,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:30,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:32,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:34,380 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:13:34,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:34,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:13:34,910 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:13:36,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:38,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:40,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:42,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:44,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:46,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:48,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:13:50,420 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:13:50,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:52,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:54,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:56,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:13:58,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:00,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:02,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:04,453 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:14:04,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:14:05,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:14:06,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:08,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:10,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:12,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:14,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:16,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:18,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:20,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:21,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:14:21,178 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:14:22,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:24,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:26,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:29,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:31,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:33,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:34,528 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:14:35,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:36,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:14:36,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:14:37,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:39,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:41,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:43,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:45,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:47,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:49,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:51,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:51,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:14:51,863 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:14:53,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:55,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:57,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:14:59,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:01,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:03,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:04,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:15:05,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:07,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:15:07,142 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:15:07,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:09,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:11,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:13,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:15,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:17,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:19,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:21,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:22,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:15:22,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:15:23,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:25,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:27,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:29,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:31,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:33,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:34,650 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:15:35,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:37,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:37,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:15:37,918 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:15:39,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:41,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:43,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:45,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:47,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:49,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:51,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:53,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:15:53,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:15:53,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:55,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:57,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:15:59,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:01,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:03,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:04,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:16:05,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:07,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:08,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:16:08,597 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:16:09,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:11,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:13,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:15,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:17,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:19,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:21,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:23,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:23,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:16:23,819 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:16:25,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:27,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:29,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:32,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:34,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:34,797 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:16:36,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:38,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:39,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:16:39,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:16:40,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:42,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:44,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:46,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:48,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:50,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:52,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:16:54,580 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:16:54,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:56,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:16:58,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:00,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:02,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:04,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:04,873 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:17:06,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:08,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:17:09,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:17:10,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:12,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:14,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:16,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:18,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:20,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:22,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:24,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:25,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:17:25,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:17:26,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:28,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:30,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:32,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:34,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:34,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:17:36,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:38,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:40,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:40,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:17:40,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:17:42,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:44,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:46,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:48,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:50,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:52,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:54,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:56,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:17:56,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:17:57,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:17:59,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:01,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:03,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:05,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:18:05,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:07,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:09,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:18:11,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:18:11,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:13,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:15,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:17,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:19,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:21,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:23,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:25,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:27,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:28,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:18:28,229 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:18:29,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:31,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:33,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:35,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:18:35,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:37,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:39,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:41,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:43,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:18:43,586 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:18:43,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:45,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:47,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:49,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:51,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:54,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:56,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:58,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:18:59,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:18:59,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:19:00,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:02,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:04,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:05,181 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:19:06,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:08,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:10,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:12,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:14,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:19:14,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:19:14,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:16,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:18,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:20,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:22,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:24,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:26,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:28,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:19:30,278 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:19:30,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:32,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:34,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:35,253 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:19:36,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:38,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:40,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:42,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:44,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:45,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:19:45,518 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:19:46,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:48,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:50,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:52,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:54,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:56,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:19:58,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:00,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:20:00,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:20:00,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:02,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:04,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:05,329 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:20:06,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:08,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:10,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:13,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:15,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:16,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:20:16,239 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:20:17,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:19,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:21,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:23,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:25,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:27,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:29,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:31,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:20:31,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:20:31,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:33,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:35,406 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:20:35,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:37,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:39,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:41,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:43,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:45,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:46,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:20:46,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:20:47,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:49,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:51,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:53,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:55,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:57,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:20:59,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:01,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:02,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:21:02,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:21:03,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:05,483 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:21:05,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:07,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:09,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:11,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:13,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:15,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:21:17,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:21:17,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:19,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:21,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:23,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:25,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:27,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:29,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:31,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:32,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:21:32,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:21:33,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:35,557 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:21:35,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:37,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:39,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:41,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:43,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:46,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:48,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:21:48,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:21:48,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:50,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:52,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:54,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:56,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:21:58,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:00,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:02,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:03,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:22:03,426 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:22:04,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:05,630 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:22:06,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:08,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:10,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:12,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:14,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:16,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:18,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:22:18,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:22:20,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:22,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:24,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:26,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:28,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:30,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:32,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:34,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:22:34,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:22:34,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:35,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:22:36,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:38,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:40,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:42,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:44,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:46,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:48,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:49,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:22:49,563 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:22:50,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:52,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:54,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:56,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:22:58,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:00,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:02,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:04,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:23:04,809 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:23:04,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:05,778 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:23:06,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:08,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:10,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:12,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:14,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:16,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:18,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:20,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:23:20,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:23:20,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:22,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:24,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:26,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:28,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:30,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:32,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:34,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:35,853 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:23:35,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:23:35,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:23:37,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:39,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:41,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:43,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:45,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:47,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:49,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:51,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:23:51,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:23:51,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:53,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:55,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:57,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:23:59,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:01,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:03,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:05,929 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:24:05,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:06,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:24:06,414 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:24:07,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:09,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:11,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:13,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:15,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:17,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:19,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:21,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:24:21,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:24:21,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:23,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:25,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:27,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:29,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:31,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:33,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:35,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:36,005 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:24:37,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:24:37,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:24:37,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:39,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:41,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:43,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:45,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:47,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:49,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:51,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:52,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:24:52,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:24:53,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:55,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:24:58,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:00,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:02,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:05,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:06,080 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:25:07,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:25:08,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:25:09,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:11,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:13,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:15,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:17,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:19,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:21,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:23,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:23,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:25:23,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:25:25,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:27,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:29,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:31,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:33,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:35,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:36,156 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:25:37,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:39,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:39,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:25:39,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:25:41,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:43,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:45,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:47,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:49,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:51,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:53,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:54,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:25:54,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:25:55,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:57,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:25:59,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:01,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:03,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:05,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:06,219 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:26:07,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:09,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:10,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:26:10,366 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:26:11,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:13,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:15,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:17,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:19,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:21,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:23,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:25,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:25,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:26:25,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:26:27,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:29,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:31,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:33,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:35,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:36,289 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:26:37,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:39,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:41,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:41,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:26:41,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:26:43,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:45,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:47,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:49,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:51,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:53,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:55,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:56,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:26:56,427 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:26:57,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:26:59,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:01,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:03,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:05,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:06,354 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:27:07,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:09,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:12,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:12,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:27:12,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:27:14,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:16,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:18,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:20,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:22,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:24,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:26,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:28,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:28,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:27:28,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:27:30,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:32,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:34,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:36,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:36,428 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:27:38,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:40,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:42,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:44,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:44,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:27:44,243 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:27:46,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:48,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:50,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:52,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:54,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:56,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:58,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:27:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:27:59,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:28:00,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:02,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:04,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:06,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:06,506 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:28:08,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:10,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:12,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:14,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:14,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:28:14,987 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:28:16,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:18,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:20,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:22,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:24,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:26,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:28,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:30,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:30,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:28:30,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:28:32,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:35,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:36,581 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:28:37,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:39,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:41,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:43,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:45,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:45,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:28:45,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:28:47,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:49,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:51,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:53,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:55,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:57,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:28:59,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:00,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:29:00,853 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:29:01,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:03,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:05,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:06,657 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:29:07,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:09,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:11,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:13,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:15,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:16,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:29:16,242 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:29:17,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:19,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:21,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:23,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:25,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:27,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:29,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:31,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:31,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:29:31,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:29:33,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:35,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:36,733 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:29:37,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:39,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:41,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:43,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:45,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:29:46,864 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:29:47,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:50,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:52,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:54,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:56,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:29:58,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:00,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:02,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:02,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:30:02,222 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:30:04,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:06,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:06,817 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:30:08,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:10,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:12,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:14,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:16,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:17,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:30:17,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:30:18,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:20,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:22,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:24,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:26,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:28,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:30,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:32,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:30:33,340 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:30:34,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:36,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:36,893 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:30:38,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:40,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:42,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:44,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:46,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:48,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:50,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:50,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:30:50,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:30:52,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:54,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:56,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:30:59,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:01,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:03,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:05,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:05,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:31:05,883 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:31:06,967 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:31:07,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:09,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:11,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:13,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:15,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:17,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:19,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:21,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:21,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:31:21,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:31:23,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:25,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:27,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:29,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:31,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:33,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:35,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:31:36,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:31:37,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:37,254 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:31:39,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:41,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:43,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:45,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:47,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:49,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:51,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:31:52,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:31:53,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:55,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:57,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:31:59,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:01,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:04,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:06,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:07,114 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:32:08,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:08,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:32:08,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:32:10,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:12,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:14,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:16,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:18,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:20,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:22,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:24,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:32:24,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:32:24,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:26,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:28,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:30,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:32,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:34,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:36,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:37,181 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:32:38,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:39,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:32:39,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:32:40,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:42,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:44,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:46,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:48,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:50,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:52,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:54,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:55,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:32:55,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:32:56,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:32:58,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:00,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:02,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:04,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:06,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:07,244 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:33:08,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:10,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:10,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:33:10,792 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:33:12,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:14,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:16,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:18,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:20,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:22,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:24,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:26,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:33:26,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:33:26,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:28,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:30,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:32,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:34,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:36,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:37,319 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:33:38,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:40,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:41,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:33:41,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:33:42,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:44,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:47,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:49,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:51,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:53,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:55,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:57,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:33:57,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:33:57,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:33:59,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:01,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:03,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:05,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:07,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:07,394 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:34:09,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:11,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:12,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:34:12,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:34:13,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:15,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:17,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:19,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:21,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:23,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:25,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:27,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:27,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:34:27,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:34:29,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:31,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:33,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:35,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:37,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:37,467 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:34:39,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:41,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:42,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:34:42,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:34:43,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:45,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:47,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:49,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:51,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:53,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:55,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:57,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:34:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:34:58,794 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:34:59,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:01,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:03,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:05,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:07,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:07,543 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:35:09,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:11,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:13,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:35:14,168 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:35:15,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:17,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:19,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:21,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:23,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:25,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:27,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:29,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:35:29,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:35:31,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:33,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:35,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:37,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:37,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:35:39,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:42,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:44,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:44,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:35:44,920 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:35:46,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:48,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:50,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:52,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:54,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:56,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:35:58,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:00,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:36:00,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:36:00,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:02,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:04,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:06,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:07,692 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:36:08,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:10,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:12,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:14,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:16,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:16,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:36:16,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:36:18,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:20,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:22,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:24,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:26,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:28,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:30,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:32,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:36:32,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:36:32,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:34,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:36,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:37,766 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:36:38,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:40,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:42,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:44,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:46,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:47,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:36:47,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:36:48,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:50,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:52,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:54,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:56,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:36:58,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:00,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:03,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:37:03,028 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:37:03,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:05,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:07,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:07,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:37:09,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:11,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:13,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:15,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:17,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:18,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:37:18,606 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:37:19,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:21,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:23,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:25,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:27,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:29,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:31,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:33,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:34,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:37:34,087 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:37:35,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:37,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:37,918 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:37:39,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:41,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:43,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:45,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:47,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:49,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:49,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:37:49,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:37:51,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:53,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:55,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:57,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:37:59,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:01,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:03,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:04,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:38:04,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:38:05,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:07,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:07,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:38:09,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:11,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:13,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:15,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:17,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:19,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:20,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:38:20,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:38:21,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:23,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:25,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:28,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:30,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:32,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:34,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:35,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:38:35,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:38:36,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:38,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:38:38,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:40,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:42,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:44,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:46,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:48,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:50,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:38:51,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:38:52,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:54,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:56,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:38:58,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:00,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:02,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:04,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:06,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:06,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:39:06,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:39:08,112 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:39:08,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:10,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:12,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:14,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:16,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:18,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:20,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:22,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:39:22,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:39:22,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:24,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:26,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:28,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:30,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:32,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:34,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:36,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:37,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:39:37,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:39:38,182 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:39:38,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:40,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:42,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:44,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:46,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:48,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:50,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:52,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:53,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:39:53,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:39:54,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:57,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:39:59,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:01,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:03,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:05,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:07,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:08,256 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:40:08,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:40:08,860 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:40:09,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:11,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:13,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:15,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:17,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:19,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:21,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:23,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:40:24,359 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:40:25,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:27,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:29,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:31,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:33,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:35,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:37,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:38,330 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:40:39,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:40:39,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:40:41,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:43,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:45,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:47,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:49,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:51,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:53,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:55,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:55,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:40:55,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:40:57,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:40:59,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:01,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:03,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:05,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:07,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:08,407 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:41:09,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:11,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:41:11,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:41:11,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:13,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:15,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:17,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:19,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:21,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:23,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:25,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:26,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:41:26,758 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:41:28,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:30,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:32,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:34,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:36,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:38,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:38,482 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:41:40,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:42,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:41:42,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:41:42,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:44,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:46,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:48,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:50,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:52,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:54,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:56,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:41:57,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:41:57,498 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:41:58,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:00,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:02,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:04,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:06,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:08,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:08,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:42:10,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:12,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:12,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:42:12,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:42:14,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:16,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:18,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:20,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:22,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:24,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:26,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:28,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:42:28,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:42:28,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:30,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:32,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:34,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:36,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:38,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:38,631 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:42:40,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:43,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:43,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:42:43,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:42:45,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:47,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:49,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:51,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:53,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:55,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:57,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:42:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:42:58,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:42:59,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:01,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:03,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:05,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:07,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:08,709 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:43:09,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:11,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:13,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:14,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:43:14,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:43:15,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:17,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:19,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:21,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:23,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:25,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:27,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:29,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:43:29,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:43:31,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:33,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:35,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:37,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:38,782 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:43:39,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:41,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:43,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:44,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:43:44,869 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:43:45,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:47,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:49,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:51,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:53,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:55,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:57,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:43:59,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:00,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:44:00,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:44:01,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:03,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:05,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:07,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:08,859 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:44:09,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:11,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:13,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:15,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:15,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:44:15,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:44:17,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:20,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:22,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:24,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:26,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:28,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:30,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:30,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:44:30,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:44:32,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:34,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:36,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:38,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:38,934 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:44:40,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:42,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:44,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:46,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:44:46,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:44:46,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:48,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:50,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:52,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:54,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:56,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:44:58,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:00,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:45:01,683 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:45:02,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:04,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:06,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:08,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:09,011 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:45:10,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:12,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:14,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:16,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:17,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:45:17,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:45:18,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:20,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:22,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:24,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:26,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:28,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:30,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:32,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:45:32,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:45:32,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:34,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:36,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:38,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:39,073 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:45:40,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:42,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:45,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:47,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:47,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:45:47,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:45:49,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:51,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:53,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:55,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:57,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:45:59,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:01,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:03,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:46:03,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:46:03,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:05,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:07,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:09,150 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:46:09,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:11,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:13,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:15,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:17,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:18,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:46:18,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:46:19,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:21,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:23,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:25,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:27,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:29,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:31,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:33,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:33,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:46:33,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:46:35,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:37,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:39,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:46:39,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:41,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:43,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:45,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:47,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:49,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:46:49,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:46:49,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:51,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:53,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:55,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:57,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:46:59,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:01,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:03,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:04,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:47:04,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:47:05,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:07,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:09,302 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:47:09,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:11,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:13,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:15,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:17,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:19,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:20,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:47:20,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:47:21,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:23,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:25,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:28,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:30,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:32,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:34,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:35,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:47:35,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:47:36,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:38,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:39,377 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:47:40,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:42,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:44,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:46,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:48,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:50,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:50,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:47:50,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:47:52,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:54,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:56,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:47:58,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:00,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:02,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:04,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:05,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:48:05,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:48:06,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:08,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:09,454 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:48:10,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:12,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:14,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:16,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:18,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:20,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:21,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:48:21,389 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:48:22,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:24,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:26,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:28,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:30,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:32,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:34,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:36,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:36,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:48:36,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:48:38,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:39,531 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:48:40,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:42,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:44,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:46,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:48,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:50,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:52,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:52,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:48:52,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:48:54,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:56,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:48:59,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:01,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:03,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:05,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:07,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:08,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:49:08,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:49:09,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:49:09,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:11,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:13,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:15,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:17,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:19,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:21,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:23,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:49:23,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:49:23,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:25,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:27,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:29,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:31,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:33,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:35,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:37,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:38,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:49:38,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:49:39,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:39,684 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:49:41,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:43,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:45,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:47,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:49,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:51,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:53,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:54,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:49:54,055 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:49:55,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:57,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:49:59,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:01,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:03,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:05,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:07,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:09,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:50:09,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:50:09,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:09,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:50:11,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:13,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:15,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:17,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:19,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:21,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:23,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:24,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:50:24,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:50:25,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:27,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:29,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:31,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:33,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:35,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:37,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:39,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:39,833 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:50:39,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:50:39,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:50:41,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:43,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:45,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:47,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:49,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:52,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:54,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:55,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:50:55,215 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:50:56,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:50:58,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:00,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:02,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:04,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:06,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:08,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:09,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:51:10,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:51:10,498 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:51:10,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:12,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:14,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:16,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:18,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:20,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:22,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:24,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:25,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:51:25,891 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:51:26,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:28,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:30,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:32,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:34,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:36,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:38,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:39,984 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:51:40,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:42,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:51:42,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:51:42,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:44,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:46,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:48,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:50,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:52,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:54,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:56,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:51:57,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:51:57,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:51:58,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:00,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:02,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:04,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:06,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:08,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:10,061 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:52:10,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:12,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:13,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:52:13,030 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:52:14,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:16,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:18,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:20,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:22,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:24,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:26,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:28,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:52:28,388 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:52:28,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:30,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:32,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:34,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:36,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:38,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:40,138 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:52:40,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:42,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:43,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:52:43,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:52:44,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:47,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:49,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:51,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:53,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:55,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:57,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:52:58,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:52:58,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:52:59,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:01,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:03,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:05,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:07,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:09,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:10,215 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:53:11,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:13,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:53:14,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:53:15,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:17,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:19,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:21,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:23,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:25,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:27,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:53:29,573 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:53:29,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:31,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:33,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:35,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:37,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:39,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:40,290 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:53:41,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:43,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:44,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:53:44,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:53:45,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:47,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:49,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:51,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:53,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:55,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:57,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:53:59,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:00,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:54:00,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:54:01,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:04,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:06,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:08,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:10,362 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:54:10,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:12,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:14,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:15,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:54:15,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:54:16,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:18,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:20,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:22,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:24,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:26,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:54:30,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:54:30,719 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:54:40,438 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:54:46,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:54:46,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:55:01,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:55:01,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:55:10,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:55:16,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:55:16,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:55:25,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:55:25,390 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 20:55:25,441 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 20:55:25,535 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 20:55:25,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 20:55:26,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:55:27,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:55:32,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:55:32,001 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:55:32,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:55:40,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:55:47,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:55:47,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:55:48,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:56:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:56:02,639 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:56:10,680 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:56:17,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:56:17,998 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:56:33,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:56:33,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:56:40,764 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:56:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:56:48,566 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:57:04,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:57:04,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:57:10,848 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:57:19,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:57:19,499 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:57:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:57:34,793 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:57:40,931 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:57:50,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:57:50,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:58:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:58:05,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:58:11,016 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:58:20,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:58:20,921 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:58:37,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:58:37,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:58:41,098 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:58:53,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:58:53,079 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:59:08,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:59:08,354 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:59:11,173 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:59:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:59:23,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:59:39,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:59:39,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:59:41,251 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 20:59:48,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:59:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:52,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:59:54,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:55,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 20:59:55,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 20:59:55,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:59:57,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 20:59:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 20:59:59,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:01,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:03,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:05,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:05,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:05,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:06,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:07,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:09,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:10,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:00:10,907 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:00:10,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:10,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:11,607 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:00:11,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:13,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:13,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:13,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:15,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:16,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:16,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:17,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:17,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:18,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:19,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:20,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:20,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:21,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:22,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:23,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:26,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:00:26,609 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:00:32,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:32,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:35,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:39,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:40,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:41,395 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:00:42,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:00:42,806 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:00:42,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:45,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:47,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:51,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:55,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:00:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:55,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:00:59,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:00,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:01:00,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:01:01,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:02,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:02,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:05,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:09,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:11,469 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:01:11,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:11,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:11,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:12,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:15,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:01:15,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:01:15,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:17,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:18,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:18,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:21,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:22,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:24,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:24,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:26,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:27,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:27,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:28,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:31,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:01:31,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:01:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:31,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:32,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:34,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:36,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:38,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:38,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:40,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:40,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:41,542 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:01:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:43,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:44,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:46,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:01:47,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:01:47,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:47,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:48,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:49,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:49,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:50,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:51,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:52,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:53,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:53,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:54,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:01:56,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:01:58,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:00,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:01,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:02:02,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:02:02,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:04,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:06,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:08,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:08,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:09,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:10,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:11,599 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:02:12,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:12,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:13,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:13,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:14,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:15,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:15,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:16,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:17,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:17,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:02:17,860 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:02:18,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:18,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:19,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:20,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:21,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:21,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:22,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:30,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:33,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:02:33,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:02:36,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:36,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:37,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:37,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:38,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:41,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:41,673 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:02:42,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:45,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:46,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:48,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:02:48,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:02:50,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:54,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:56,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:02:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:02:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:00,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:02,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:03,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:03,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:04,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:03:04,272 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:03:06,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:06,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:08,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:11,746 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:03:12,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:14,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:14,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:15,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:16,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:16,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:17,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:18,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:18,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:03:19,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:03:21,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:22,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:22,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:23,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:24,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:25,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:27,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:28,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:28,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:29,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:31,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:31,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:33,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:35,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:03:35,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:03:35,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:37,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:38,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:38,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:39,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:40,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:41,666 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 21:03:41,868 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 21:03:41,937 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 21:03:41,939 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:03:41,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 21:03:41,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:42,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:43,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:45,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:46,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:46,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:47,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:48,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:48,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:49,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:50,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:50,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:03:50,606 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:03:51,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:52,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:52,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:53,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:55,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:56,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:56,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:03:57,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:03:59,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:00,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:01,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:03,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:04,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:04,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:05,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:04:06,050 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:04:06,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:06,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:07,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:08,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:08,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:09,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:10,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:10,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:11,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:12,007 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:04:12,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:12,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:13,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:14,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:16,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:21,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:04:21,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:04:26,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:30,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:32,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:34,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:35,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:37,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:04:37,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:04:38,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:42,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:42,079 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:04:46,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:47,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:50,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:52,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:52,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:04:52,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:04:52,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:52,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:53,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:04:56,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:04:58,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:02,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:04,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:05,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:08,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:08,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:05:08,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:05:10,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:12,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:12,153 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:05:12,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:15,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:16,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:17,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:17,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:18,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:21,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:22,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:22,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:23,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:23,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:05:23,580 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:05:25,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:25,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:27,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:29,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:29,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:30,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:31,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:33,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:33,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:33,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:34,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:35,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:36,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:36,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:38,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:39,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:05:39,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:05:39,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:41,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:42,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:05:43,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:43,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:43,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:47,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:47,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:49,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:49,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:51,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:53,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:54,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:05:54,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:05:55,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:55,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:55,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:57,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:05:58,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:58,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:05:59,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:01,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:02,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:02,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:03,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:03,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:05,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:05,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:07,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:07,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:08,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:09,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:09,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:06:09,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:06:11,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:12,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:12,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:06:19,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:19,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:25,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:25,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:06:25,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:06:29,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:33,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:37,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:37,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:38,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:40,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:06:40,523 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:06:41,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:41,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:42,369 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:06:45,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:45,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:45,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:47,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:48,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:48,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:50,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:51,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:53,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:53,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:53,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:54,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:06:55,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:06:55,829 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:06:57,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:06:59,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:03,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:05,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:05,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:07,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:08,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:08,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:07:11,065 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:07:11,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:12,443 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:07:12,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:12,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:13,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:13,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:15,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:17,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:19,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:21,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:23,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:23,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:23,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:24,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:25,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:26,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:07:26,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:07:27,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:27,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:28,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:30,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:30,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:31,948 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 21:07:32,152 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 21:07:32,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:32,226 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 21:07:33,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 21:07:33,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:33,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:35,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:36,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:37,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:39,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:41,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:41,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:41,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:07:41,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:07:42,552 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:07:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:43,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:44,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:44,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:45,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:46,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:47,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:48,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:50,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:51,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:51,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:52,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:54,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:56,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:56,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:56,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:07:56,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:07:58,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:07:59,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:07:59,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:00,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:02,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:03,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:03,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:04,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:06,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:12,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:08:12,172 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:08:12,626 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:08:12,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:12,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:13,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:15,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:16,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:18,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:18,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:22,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:26,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:27,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:08:27,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:08:27,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:27,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:30,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:34,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:38,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:41,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:42,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:42,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:08:42,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:08:43,016 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:08:43,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:44,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:45,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:45,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:46,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:48,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:50,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:50,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:52,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:52,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:08:54,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:56,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:08:58,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:08:58,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:08:58,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:00,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:02,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:06,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:07,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:07,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:08,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:10,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:10,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:10,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:12,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:12,770 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:09:13,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:09:13,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:09:13,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:14,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:15,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:15,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:17,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:18,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:20,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:22,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:24,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:24,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:26,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:27,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:27,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:09:28,602 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:09:28,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:28,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:30,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:31,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:32,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:33,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:33,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:34,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:36,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:37,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:37,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:38,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:40,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:41,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:41,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:42,848 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:09:43,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:43,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:09:43,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:09:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:45,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:47,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:47,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:48,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:48,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:49,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:51,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:51,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:53,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:55,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:57,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:58,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:58,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:09:59,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:09:59,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:09:59,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:10:00,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:00,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:01,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:03,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:12,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:12,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:10:13,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:14,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:14,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:10:14,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:10:17,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:21,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:21,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:21,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:25,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:26,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:29,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:29,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:10:29,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:10:33,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:34,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:34,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:37,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:39,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:42,994 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:10:43,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:44,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:44,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:10:45,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:10:45,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:45,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:45,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:49,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:51,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:51,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:52,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:54,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:55,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:57,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:58,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:58,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:10:59,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:10:59,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:11:00,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:11:03,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:05,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:07,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:09,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:12,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:12,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:13,069 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:11:13,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:14,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:14,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:15,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:15,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:11:15,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:11:16,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:16,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:17,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:19,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:21,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:22,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:22,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:23,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,182 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 21:11:24,398 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 21:11:24,459 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 21:11:24,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:24,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 21:11:25,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:25,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:26,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:26,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:27,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:29,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:30,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:11:30,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:11:31,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:31,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:31,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:33,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:35,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:36,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:36,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:37,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:39,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:39,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:39,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:40,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:41,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:43,146 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:11:43,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:45,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:45,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:11:46,262 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:11:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:47,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:48,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:50,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:52,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:54,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:54,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:55,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:55,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:11:56,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:11:58,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:01,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:12:01,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:12:07,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:07,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:08,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:12,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:13,219 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:12:16,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:12:17,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:12:20,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:20,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:21,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:22,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:22,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:24,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:28,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:28,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:30,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:32,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:32,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:12:32,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:12:36,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:37,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:38,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:41,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:41,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:42,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:43,291 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:12:44,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:47,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:47,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:48,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:12:48,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:12:48,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:50,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:50,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:50,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:52,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:53,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:54,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:54,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:57,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:58,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:58,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:12:59,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:12:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:03,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:03,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:13:03,432 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:13:05,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:07,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:09,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:11,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:11,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:11,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:13,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:13:13,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:15,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:15,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:15,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:17,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:17,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:18,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:13:18,758 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:13:19,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:19,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:19,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:19,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:19,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:20,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:21,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:23,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:23,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:24,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:25,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:26,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:27,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:28,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:29,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:30,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:30,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:31,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:33,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:33,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:13:33,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:13:33,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:33,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:34,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:35,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:35,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:36,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:37,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:39,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:39,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:41,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:42,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:42,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:43,436 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:13:43,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:44,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:44,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:45,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:46,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:49,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:13:49,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:13:49,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:51,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:51,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:51,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:53,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:55,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:13:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:13:59,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:00,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:03,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:04,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:14:04,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:14:09,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:13,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:13,509 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:14:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:15,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:16,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:17,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:19,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:20,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:14:20,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:14:23,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:25,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:25,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:27,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:28,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:28,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:30,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:31,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:32,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:33,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:35,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:14:35,722 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:14:37,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:37,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:39,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:40,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:43,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:43,576 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:14:45,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:47,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:49,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:49,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:50,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:14:50,993 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:14:51,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:51,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:53,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:53,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:53,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:55,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:57,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:14:58,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:14:59,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:00,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:03,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:05,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:06,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:15:06,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:15:06,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:06,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:07,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:09,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:09,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:10,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:11,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:13,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:13,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:15:13,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:13,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:14,353 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 21:15:14,593 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 21:15:14,663 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 21:15:15,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 21:15:15,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:16,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:17,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:19,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:21,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:21,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:15:21,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:15:23,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:23,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:24,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:24,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:25,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:25,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:26,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:27,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:29,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:29,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:30,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:31,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:31,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:32,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:33,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:33,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:34,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:35,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:36,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:37,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:15:37,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:15:38,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:40,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:41,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:42,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:43,794 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:15:44,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:46,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:48,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:50,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:15:53,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:15:58,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:15:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:15:58,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:01,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:04,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:08,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:09,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:11,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:16:11,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:16:12,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:13,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:16:16,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:19,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:19,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:20,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:24,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:26,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:26,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:16:26,483 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:16:26,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:29,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:29,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:30,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:32,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:33,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:33,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:35,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:36,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:38,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:39,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:39,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:16:42,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:16:42,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:42,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:43,935 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:16:44,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:46,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:50,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:51,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:52,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:54,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:55,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:56,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:57,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:16:57,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:16:58,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:58,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:16:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:16:59,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:00,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:01,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:01,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:04,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:06,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:06,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:08,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:08,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:09,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:09,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:09,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:09,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:10,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:12,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:12,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:17:12,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:17:13,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:14,007 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:17:14,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:15,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:15,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:16,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:18,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:20,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:20,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:22,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:23,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:24,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:25,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:27,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:28,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:17:28,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:17:28,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:29,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:30,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:31,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:31,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:32,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:33,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:33,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:34,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:36,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:37,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:37,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:38,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:40,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:43,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:43,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:17:43,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:17:45,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:17:45,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:53,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:17:53,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:53,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:17:59,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:00,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:18:00,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:18:03,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:03,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:03,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:07,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:11,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:12,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:12,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:14,136 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:18:15,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:16,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:18:16,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:18:19,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:20,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:20,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:21,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:25,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:28,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:29,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:18:31,275 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:18:31,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:35,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:37,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:37,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:37,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:39,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:41,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:43,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:44,201 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:18:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:44,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:45,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:46,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:18:46,816 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:18:49,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:51,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:52,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:53,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:54,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:18:57,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:18:59,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:01,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:01,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:02,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:19:02,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:19:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:03,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:05,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:05,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:07,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 21:19:08,443 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 21:19:08,654 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 21:19:08,715 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 21:19:09,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 21:19:09,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:11,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:13,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:14,328 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:19:15,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:17,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:19:17,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:19:17,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:19,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:21,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:23,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:25,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:27,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:30,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:32,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:33,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:19:33,279 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:19:34,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:36,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:38,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:40,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:42,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:44,398 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:19:44,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:46,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:48,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:19:48,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:19:48,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:50,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:52,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:54,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:56,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:19:58,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:00,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:02,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:04,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:20:04,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:20:04,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:06,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:08,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:10,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:12,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:14,474 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:20:14,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:16,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:18,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:19,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:20:19,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:20:20,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:22,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:24,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:26,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:28,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:30,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:32,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:34,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:20:34,724 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:20:34,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:36,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:38,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:40,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:42,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:44,549 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:20:44,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:46,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:48,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:50,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:20:50,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:20:50,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:52,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:54,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:56,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:20:58,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:00,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:02,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:04,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:05,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:21:05,815 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:21:06,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:08,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:11,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:13,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:14,622 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:21:15,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:17,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:19,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:21,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:21:21,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:21:21,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:23,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:25,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:27,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:29,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:31,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:33,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:35,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:21:36,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:21:37,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:39,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:41,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:43,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:44,696 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:21:45,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:47,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:49,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:51,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:52,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:21:52,364 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:21:53,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:55,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:57,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:21:59,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:01,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:03,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:05,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:22:07,710 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:22:07,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:09,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:11,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:13,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:14,774 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:22:16,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:18,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:20,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:22,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:23,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:22:23,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:22:24,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:26,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:28,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:30,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:32,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:34,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:36,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:38,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:22:38,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:22:38,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:40,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:42,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:44,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:44,851 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:22:46,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:48,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:50,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:52,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:54,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:54,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:22:54,867 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:22:56,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:22:58,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:00,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:02,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:04,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:06,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:08,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:10,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:23:10,416 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:23:10,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:12,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:14,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:14,918 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:23:16,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:18,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:20,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:22,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:24,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:25,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:23:25,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:23:26,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:28,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:30,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:32,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:34,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:37,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:39,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:41,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:23:41,109 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:23:41,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:43,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:44,993 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:23:45,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:47,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:49,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:51,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:53,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:55,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:56,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:23:56,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:23:57,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:23:59,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:01,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:03,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:05,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:07,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:09,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:11,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:12,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:24:12,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:24:13,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:15,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:24:15,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:17,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:19,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:21,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:23,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:25,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:27,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:24:27,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:24:27,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:29,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:31,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:33,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:35,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:37,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:39,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:41,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:43,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:24:43,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:24:43,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:45,128 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:24:45,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:47,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:49,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:51,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:53,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:55,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:58,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:24:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:24:59,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:25:00,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:02,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:04,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:06,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:08,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:10,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:12,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:14,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:15,202 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:25:15,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:25:15,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:25:16,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:18,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:20,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:22,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:24,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:26,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:28,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:30,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:25:30,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:25:30,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:32,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:34,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:36,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:38,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:40,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:42,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:44,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:45,276 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:25:46,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:25:46,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:25:46,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:48,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:50,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:52,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:54,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:56,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:25:58,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:00,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:01,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:26:01,529 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:26:02,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:04,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:06,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:08,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:10,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:12,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:14,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:15,350 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:26:16,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:16,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:26:16,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:26:18,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:20,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:22,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:25,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:27,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:29,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:31,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:32,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:26:32,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:26:33,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:35,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:37,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:39,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:41,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:43,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:45,425 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:26:45,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:47,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:26:47,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:26:47,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:49,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:51,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:53,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:55,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:57,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:26:59,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:01,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:27:02,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:27:03,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:05,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:07,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:09,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:11,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:13,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:15,495 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:27:15,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:17,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:18,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:27:18,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:27:19,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:21,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:23,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:25,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:27,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:29,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:31,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:33,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:27:33,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:27:33,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:35,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:37,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:39,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:41,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:43,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:45,566 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:27:45,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:47,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:48,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:27:48,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:27:49,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:51,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:53,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:55,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:57,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:27:59,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:01,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:03,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:04,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:28:04,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:28:05,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:07,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:09,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:11,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:13,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:15,636 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:28:15,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:17,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:19,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:28:19,536 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:28:19,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:21,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:23,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:26,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:28,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:30,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:32,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:34,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:28:34,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:28:34,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:36,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:38,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:40,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:42,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:44,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:45,710 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:28:46,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:48,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:50,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:28:50,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:28:50,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:52,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:54,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:56,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:28:58,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:00,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:02,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:04,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:05,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:29:05,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:29:06,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:08,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:10,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:12,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:14,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:15,788 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:29:16,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:18,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:20,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:29:20,907 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:29:20,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:22,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:24,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:26,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:28,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:30,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:32,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:34,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:36,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:29:36,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:29:36,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:38,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:40,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:42,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:44,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:45,865 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:29:46,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:48,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:50,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:52,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:29:52,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:29:52,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:54,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:56,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:29:58,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:00,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:02,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:04,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:06,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:07,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:30:07,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:30:08,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:10,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:12,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:14,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:15,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:30:16,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:18,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:20,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:22,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:24,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:26,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:27,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:30:27,467 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:30:28,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:30,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:33,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:35,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:37,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:39,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:41,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:43,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:30:43,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:30:43,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:45,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:46,018 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:30:48,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:50,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:52,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:54,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:56,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:30:58,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:00,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:01,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:31:01,098 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:31:02,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:04,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:06,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:08,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:10,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:12,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:14,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:16,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:16,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:31:16,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:31:16,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:31:18,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:20,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:22,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:24,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:26,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:28,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:30,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:32,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:31:32,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:31:34,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:36,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:38,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:40,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:42,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:44,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:46,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:46,170 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:31:47,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:31:47,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:31:48,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:50,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:53,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:55,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:57,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:31:59,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:01,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:32:03,006 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:32:03,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:05,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:07,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:09,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:11,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:13,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:15,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:16,247 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:32:17,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:18,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:32:18,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:32:19,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:21,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:23,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:25,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:27,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:29,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:31,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:33,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:33,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:32:33,997 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:32:35,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:37,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:39,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:41,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:43,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:45,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:46,326 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:32:47,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:49,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:49,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:32:49,401 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:32:51,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:53,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:55,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:57,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:32:59,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:01,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:03,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:05,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:06,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:33:06,381 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:33:07,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:10,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:12,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:14,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:16,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:16,401 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:33:18,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:20,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:21,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:33:21,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:33:22,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:24,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:26,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:28,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:30,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:32,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:34,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:36,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:37,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:33:37,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:33:38,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:40,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:42,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:44,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:46,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:46,477 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:33:48,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:50,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:52,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:52,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:33:52,708 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:33:54,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:56,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:33:58,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:00,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:02,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:04,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:06,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:08,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:34:08,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:34:08,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:10,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:12,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:14,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:16,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:16,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:34:18,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:20,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:22,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:23,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:34:23,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:34:24,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:26,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:28,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:30,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:32,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:34,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:36,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:38,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:39,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:34:39,179 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:34:40,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:42,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:44,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:46,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:46,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:34:48,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:50,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:52,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:54,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:34:54,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:34:56,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:34:58,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:01,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:03,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:05,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:07,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:09,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:10,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:35:10,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:35:11,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:13,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:15,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:16,701 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:35:17,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:19,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:21,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:23,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:25,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:25,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:35:25,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:35:27,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:29,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:31,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:33,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:35,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:37,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:39,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:41,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:41,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:35:41,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:35:43,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:45,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:46,776 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:35:47,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:49,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:51,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:53,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:55,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:56,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:35:56,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:35:57,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:35:59,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:01,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:03,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:05,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:07,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:09,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:11,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:12,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:36:12,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:36:13,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:15,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:16,853 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:36:17,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:19,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:21,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:23,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:25,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:27,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:27,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:36:27,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:36:29,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:32,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:34,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:36,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:38,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:40,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:42,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:36:42,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:36:44,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:46,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:46,929 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:36:48,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:50,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:52,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:54,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:56,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:36:58,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:36:58,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:36:58,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:00,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:02,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:04,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:06,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:08,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:10,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:12,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:13,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:37:13,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:37:14,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:16,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:17,009 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:37:18,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:20,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:22,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:24,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:26,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:28,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:28,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:37:28,813 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:37:30,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:32,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:34,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:36,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:38,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:40,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:42,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:37:44,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:37:44,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:47,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:37:47,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:49,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:51,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:53,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:55,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:57,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:59,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:37:59,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:37:59,423 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:38:01,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:03,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:05,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:07,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:09,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:11,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:13,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:14,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:38:14,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:38:15,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:17,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:38:17,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:19,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:21,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:23,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:25,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:27,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:29,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:38:30,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:38:31,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:33,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:35,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:37,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:39,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:41,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:43,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:45,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:45,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:38:45,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:38:47,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:47,232 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:38:49,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:51,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:53,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:55,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:57,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:38:59,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:00,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:39:00,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:39:01,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:03,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:05,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:07,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:09,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:11,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:13,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:15,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:16,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:39:16,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:39:17,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:17,311 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:39:19,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:21,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:23,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:25,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:27,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:29,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:31,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:31,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:39:31,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:39:33,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:35,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:37,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:39,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:41,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:43,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:45,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:47,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:47,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:39:47,315 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:39:47,591 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:39:50,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:52,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:54,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:56,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:39:58,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:00,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:02,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:40:02,592 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:40:04,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:06,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:08,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:10,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:12,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:14,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:16,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:17,459 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:40:18,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:40:18,020 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:40:18,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:20,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:22,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:24,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:26,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:28,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:30,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:32,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:33,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:40:33,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:40:34,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:36,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:38,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:40,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:42,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:44,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:46,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:47,535 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:40:48,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:48,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:40:48,721 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:40:50,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:52,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:54,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:56,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:40:58,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:00,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:02,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:04,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:41:04,120 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:41:04,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:06,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:08,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:10,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:12,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:14,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:16,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:17,609 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:41:19,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:19,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:41:19,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:41:21,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:23,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:25,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:27,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:29,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:31,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:33,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:34,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:41:34,765 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:41:35,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:37,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:39,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:41,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:43,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:45,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:47,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:47,683 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:41:49,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:50,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:41:50,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:41:51,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:53,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:55,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:57,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:41:59,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:01,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:03,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:05,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:05,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:42:05,677 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:42:07,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:09,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:11,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:13,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:15,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:17,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:17,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:42:19,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:20,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:42:20,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:42:21,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:23,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:25,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:27,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:29,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:31,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:33,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:35,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:36,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:42:36,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:42:37,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:39,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:41,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:43,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:45,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:47,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:47,832 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:42:49,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:42:51,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:42:52,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:54,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:56,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:42:58,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:00,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:02,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:04,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:06,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:06,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:43:06,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:43:08,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:10,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:12,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:14,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:16,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:17,909 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:43:18,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:20,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:22,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:43:22,092 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:43:22,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:24,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:26,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:28,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:30,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:32,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:34,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:36,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:43:37,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:43:38,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:40,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:42,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:44,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:46,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:47,981 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:43:48,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:50,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:52,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:52,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:43:52,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:43:54,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:56,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:43:58,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:00,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:02,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:04,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:06,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:08,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:44:08,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:44:08,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:10,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:12,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:14,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:16,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:18,055 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:44:18,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:20,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:22,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:23,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:44:23,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:44:24,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:26,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:29,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:31,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:33,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:35,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:37,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:38,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:44:38,825 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:44:39,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:41,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:43,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:45,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:47,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:48,129 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:44:49,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:51,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:53,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:54,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:44:54,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:44:55,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:57,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:44:59,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:01,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:03,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:05,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:07,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:09,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:45:09,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:45:11,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:13,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:15,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:17,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:18,207 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:45:19,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:21,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:23,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:24,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:45:24,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:45:25,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:27,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:29,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:31,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:33,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:35,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:37,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:39,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:40,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:45:40,149 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:45:41,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:43,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:45,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:47,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:48,283 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:45:49,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:51,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:53,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:55,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:55,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:45:55,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:45:57,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:45:59,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:01,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:04,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:06,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:08,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:10,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:46:10,781 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:46:12,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:14,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:16,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:18,360 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:46:18,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:20,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:22,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:24,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:26,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:46:26,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:46:26,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:28,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:30,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:32,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:34,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:36,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:38,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:40,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:41,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:46:41,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:46:42,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:44,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:46,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:48,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:48,433 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:46:50,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:52,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:54,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:56,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:46:56,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:46:56,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:46:58,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:00,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:02,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:04,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:06,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:08,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:10,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:12,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:47:12,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:47:12,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:14,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:16,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:18,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:18,510 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:47:20,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:22,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:24,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:26,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:27,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:47:27,445 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:47:28,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:30,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:32,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:34,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:36,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:38,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:40,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:42,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:42,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:47:42,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:47:44,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:46,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:48,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:48,588 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:47:50,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:52,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:55,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:57,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:47:57,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:47:57,986 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:47:59,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:01,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:03,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:05,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:07,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:09,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:11,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:13,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:48:13,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:48:13,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:15,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:17,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:18,661 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:48:19,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:21,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:23,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:25,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:27,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:28,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:48:28,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:48:29,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:31,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:33,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:35,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:37,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:39,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:41,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:43,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:48:44,042 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:48:45,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:47,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:48,736 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:48:49,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:51,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:53,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:55,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:57,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:59,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:48:59,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:48:59,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:49:01,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:03,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:05,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:07,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:10,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:12,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:14,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:14,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:49:14,778 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:49:16,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:18,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:18,813 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:49:20,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:22,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:24,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:26,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:28,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:30,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:49:30,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:49:30,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:32,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:34,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:36,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:38,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:40,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:42,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:44,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:45,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:49:45,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:49:46,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:48,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:48,889 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:49:50,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:52,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:54,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:56,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:49:58,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:00,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:00,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:50:00,610 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:50:02,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:04,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:06,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:08,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:10,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:12,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:14,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:15,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:50:15,898 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:50:16,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:18,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:18,965 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:50:20,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:22,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:24,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:26,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:29,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:31,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:50:31,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:50:31,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:33,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:35,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:37,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:39,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:41,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:43,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:45,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:46,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:50:46,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:50:47,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:49,039 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:50:49,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:51,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:53,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:55,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:57,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:50:59,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:01,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:01,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:51:01,988 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:51:03,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:05,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:07,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:09,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:11,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:13,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:15,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:17,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:51:17,344 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:51:17,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:19,115 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:51:19,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:21,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:23,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:25,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:27,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:29,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:31,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:32,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:51:32,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:51:33,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:35,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:37,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:39,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:41,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:43,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:45,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:47,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:51:47,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:51:49,190 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:51:49,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:51,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:53,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:55,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:57,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:51:59,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:01,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:03,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:52:03,176 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:52:03,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:05,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:07,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:09,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:12,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:14,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:16,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:18,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:52:18,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:52:18,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:19,272 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:52:20,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:22,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:24,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:26,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:28,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:30,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:32,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:33,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:52:33,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:52:34,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:36,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:38,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:40,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:42,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:44,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:46,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:48,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:49,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:52:49,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:52:49,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:52:50,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:52,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:54,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:56,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:52:58,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:00,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:02,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:04,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:04,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:53:04,674 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:53:06,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:08,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:10,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:12,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:14,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:16,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:18,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:19,433 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:53:20,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:53:20,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:53:20,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:22,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:24,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:26,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:28,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:31,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:33,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:53:35,334 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:53:35,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:37,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:39,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:41,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:43,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:45,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:47,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:49,511 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:53:49,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:50,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:53:50,605 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:53:51,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:53,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:55,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:57,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:53:59,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:01,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:03,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:05,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:05,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:54:05,954 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:54:07,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:09,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:11,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:13,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:15,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:17,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:19,587 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:54:19,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:21,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:54:21,207 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:54:21,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:23,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:25,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:27,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:29,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:31,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:33,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:35,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:36,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:54:36,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:54:37,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:39,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:41,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:43,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:45,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:47,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:49,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:49,661 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:54:51,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:51,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:54:51,923 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:54:53,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:55,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:57,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:54:59,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:01,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:03,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:05,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:07,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:55:07,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:55:07,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:09,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:11,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:13,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:15,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:18,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:19,738 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:55:20,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:22,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:55:22,570 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:55:22,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:24,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:26,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:28,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:30,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:32,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:34,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:36,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:38,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:55:38,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:55:38,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:40,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:42,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:44,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:46,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:48,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:49,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:55:50,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:52,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:55:53,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:55:54,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:56,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:55:58,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:00,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:02,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:04,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:06,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:08,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:08,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:56:08,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:56:10,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:12,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:14,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:16,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:18,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:19,892 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:56:20,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:22,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:24,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:56:24,248 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:56:24,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:26,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:28,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:30,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:32,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:34,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:36,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:38,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:56:39,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:56:40,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:43,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:45,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:47,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:49,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:49,968 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:56:51,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:53,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:54,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:56:54,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:56:55,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:57,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:56:59,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:01,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:03,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:05,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:07,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:09,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:10,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:57:10,085 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:57:11,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:13,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:15,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:17,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:19,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:20,046 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:57:21,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:23,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:25,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:57:25,321 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:57:25,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:27,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:29,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:31,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:33,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:35,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:37,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:39,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:40,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:57:40,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:57:41,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:43,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:45,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:47,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:49,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:50,121 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:57:51,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:53,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:55,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:56,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:57:56,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:57:57,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:57:59,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:01,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:03,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:05,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:07,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:09,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:11,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:58:11,245 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:58:11,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:13,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:15,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:17,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:19,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:20,202 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:58:21,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:23,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:25,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:26,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:58:26,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:58:27,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:29,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:31,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:33,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:35,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:37,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:39,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:41,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:41,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:58:41,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:58:43,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:46,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:48,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:50,274 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:58:50,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:52,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:54,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:56,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:58:57,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:58:57,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:58:58,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:00,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:02,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:04,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:06,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:08,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:10,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:59:12,319 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:59:12,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:14,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:16,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:18,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:20,346 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:59:20,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:22,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:24,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:26,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:27,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:59:27,641 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:59:28,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:30,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:32,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:34,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:36,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:38,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:40,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:42,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:59:42,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:59:44,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:46,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:48,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:50,423 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 21:59:50,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:52,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:54,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:56,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 21:59:58,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 21:59:58,174 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 21:59:58,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:00,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:02,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:04,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:06,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:08,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:10,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:12,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:13,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:00:13,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:00:15,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:17,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:19,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:20,498 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:00:21,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:23,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:25,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:27,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:00:28,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:00:29,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:31,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:33,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:35,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:37,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:39,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:41,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:43,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:44,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:00:44,892 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:00:45,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:47,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:49,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:50,575 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:00:51,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:53,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:55,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:57,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:00:59,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:01,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:01:01,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:01:01,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:03,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:05,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:07,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:09,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:11,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:01:17,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:01:17,519 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:01:20,653 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:01:32,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:01:32,913 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:01:48,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:01:48,196 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:01:50,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:02:03,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:02:03,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:02:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:02:10,770 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 22:02:10,823 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 22:02:10,901 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 22:02:11,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 22:02:11,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:02:12,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:02:17,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:02:19,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:02:19,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:02:20,931 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:02:33,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:02:34,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:02:34,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:02:49,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:02:49,861 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:02:51,013 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:03:05,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:03:05,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:03:20,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:03:20,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:03:21,096 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:03:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:03:35,730 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:03:51,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:03:51,091 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:03:51,375 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:04:06,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:04:06,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:04:21,263 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:04:21,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:04:21,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:04:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:04:36,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:04:51,345 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:04:52,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:04:52,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:05:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:05:07,390 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:05:21,427 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:05:22,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:05:22,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:05:37,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:05:37,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:05:51,505 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:05:53,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:05:53,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:06:09,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:06:09,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:06:21,580 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:06:24,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:06:24,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:06:35,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:37,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:39,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:40,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:06:40,067 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:06:40,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:40,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:41,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:43,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:45,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:45,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:48,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:48,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:50,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:50,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:51,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:51,650 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:06:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:52,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:53,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:53,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:54,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:54,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:06:55,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:06:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:56,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:58,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:58,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:06:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:06:59,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:00,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:02,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:02,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:04,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:06,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:06,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:07,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:07,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:08,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:10,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:07:10,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:07:10,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:18,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:21,722 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:07:24,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:07:26,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:07:28,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:31,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:32,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:36,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:37,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:37,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:38,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:40,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:42,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:07:42,248 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:07:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:42,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:44,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:46,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:48,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:48,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:50,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:51,795 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:07:52,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:56,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:56,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:07:57,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:07:57,505 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:07:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:57,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:07:58,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:00,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:00,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:02,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:04,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:06,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:08,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:08,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:10,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:11,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:11,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:12,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:13,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:08:13,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:08:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:14,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:15,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:18,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:19,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:19,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:21,866 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:08:22,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:22,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:24,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:24,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:25,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:26,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:28,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:28,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:08:28,638 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:08:29,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:29,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:30,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:31,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:32,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:33,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:33,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:34,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:34,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:36,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:36,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:38,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:40,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:40,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:40,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:41,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:42,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:44,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:44,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:44,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:08:44,348 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:08:45,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:45,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:46,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:48,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:48,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:48,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:50,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:50,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:51,936 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:08:52,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:52,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:52,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:53,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:54,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:56,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:58,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:08:59,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:08:59,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:08:59,666 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:09:00,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:00,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:01,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:02,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:02,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:02,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:02,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:02,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:02,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:02,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:03,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:04,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:06,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:09,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:09,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:13,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:15,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:09:15,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:09:16,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:20,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:22,013 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:09:24,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:28,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:09:31,373 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:09:31,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:31,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:32,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:35,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:35,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:36,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:39,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:40,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:41,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:42,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:46,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:47,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:09:47,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:09:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:47,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:48,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:52,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:52,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:09:54,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:56,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:09:56,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:56,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:09:59,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:00,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:02,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:02,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:10:02,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:10:03,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:03,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:04,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:04,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:05,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:06,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:08,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:08,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:09,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:10,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:10,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:12,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:13,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:14,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:16,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:16,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:16,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:18,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:10:18,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:10:18,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:20,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:21,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:21,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:22,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:10:22,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:22,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:24,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:25,144 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 22:10:25,361 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 22:10:25,426 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 22:10:26,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 22:10:26,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:27,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:27,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:28,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:30,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:30,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:30,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:32,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:33,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:10:33,905 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:10:34,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:34,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:36,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:36,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:38,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:38,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:40,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:41,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:43,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:45,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:46,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:47,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:48,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:49,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:49,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:10:49,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:10:51,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:52,292 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:10:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:53,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:54,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:54,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:55,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:55,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:57,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:59,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:10:59,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:10:59,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:11:06,273 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:11:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:07,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:11,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:13,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:13,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:15,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:17,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:19,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:22,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:11:22,100 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:11:22,411 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:11:23,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:27,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:29,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:29,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:31,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:32,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:33,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:34,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:34,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:37,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:11:37,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:11:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:37,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:39,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:43,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:45,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:49,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:50,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:51,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:51,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:52,437 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:11:53,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:11:53,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:11:53,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:53,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:55,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:57,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:59,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:11:59,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:11:59,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:01,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:02,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:02,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:03,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:05,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:06,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:06,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:07,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:12:08,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:12:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:08,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:09,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:11,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:12,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:12,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:13,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:15,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:15,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:15,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:16,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:17,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:17,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:19,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:20,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:22,509 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:12:22,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:22,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:24,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:12:24,210 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:12:24,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:24,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:26,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:28,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:28,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:28,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:29,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:30,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:30,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:32,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:33,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:34,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:35,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:35,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:36,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:37,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:37,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:38,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:38,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:39,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:12:39,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:12:40,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:41,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:42,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:43,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:43,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:44,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:45,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:45,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:46,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:48,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:50,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:50,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:50,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:12:52,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:52,577 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:12:54,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:12:54,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:12:54,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:13:02,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:07,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:07,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:08,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:10,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:13:10,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:13:12,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:12,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:16,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:18,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:18,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:20,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:21,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:22,645 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:13:24,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:27,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:13:27,059 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:13:28,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:30,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:33,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:34,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:36,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:37,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:37,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:38,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:40,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:42,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:42,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:13:42,560 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:13:46,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:48,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:50,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:51,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:51,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,711 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:13:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:52,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:54,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:55,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:13:56,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:13:58,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:13:58,090 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:13:58,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:00,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:02,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:04,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:05,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:05,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:06,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:08,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:08,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:09,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:10,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:11,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:11,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:12,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:13,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:13,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:14:13,450 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:14:14,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:14,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:14,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:14,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:15,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:15,469 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 22:14:15,691 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 22:14:15,761 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 22:14:16,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 22:14:17,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:18,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:18,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:19,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:21,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:22,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:22,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:14:23,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:25,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:25,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:26,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:27,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:29,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:14:29,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:14:29,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:29,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:29,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:30,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:31,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:32,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:33,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:34,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:35,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:37,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:37,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:39,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:41,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:41,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:41,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:42,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:43,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:43,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:14:44,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:14:44,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:45,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:46,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:47,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:47,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:49,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:52,949 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:14:56,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:56,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:14:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:14:59,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:14:59,899 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:15:05,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:08,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:08,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:09,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:10,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:10,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:12,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:13,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:15,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:15:15,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:15:17,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:17,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:21,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:22,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:22,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:23,025 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:15:23,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:26,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:26,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:27,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:30,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:15:30,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:15:31,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:33,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:33,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:33,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:37,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:37,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:40,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:42,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:44,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:46,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:46,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:15:46,675 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:15:50,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:52,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:53,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:15:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:53,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:15:54,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:15:58,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:00,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:01,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:16:01,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:16:02,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:04,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:04,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:05,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:05,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:06,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:08,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:10,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:10,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:10,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:11,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:12,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:14,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:14,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:16,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:16,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:18,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:16:18,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:16:18,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:20,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:20,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:20,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:22,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:23,162 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:16:23,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:23,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:24,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:26,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:27,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:27,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:28,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:28,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:30,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:31,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:32,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:33,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:16:33,612 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:16:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:33,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:34,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:35,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:36,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:36,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:37,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:38,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:40,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:42,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:42,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:43,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:44,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:44,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:46,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:46,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:16:48,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:16:48,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:16:48,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:16:53,230 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:16:56,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:02,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:03,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:03,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:04,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:17:04,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:17:06,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:09,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:09,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:10,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:13,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:14,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:16,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:17:19,616 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:17:20,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:22,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,305 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:17:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:23,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:24,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:26,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:30,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:32,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:33,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:33,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:35,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:17:35,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:17:37,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:39,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:39,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:39,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:40,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:41,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:45,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:47,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:47,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:49,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:50,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:17:50,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:17:50,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:50,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:51,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:53,379 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:17:55,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:55,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:57,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:17:57,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:17:59,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:01,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:01,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:03,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:05,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:05,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:05,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:18:05,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:18:06,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:06,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,010 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 22:18:07,224 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 22:18:07,282 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 22:18:07,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 22:18:07,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:07,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:07,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:08,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:09,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:11,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:12,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:13,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:15,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:16,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:16,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:17,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:17,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:19,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:20,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:20,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:18:20,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:18:21,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:23,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:23,453 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:18:23,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:23,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:25,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:25,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:26,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:27,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:29,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:29,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:29,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:30,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:31,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:31,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:33,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:35,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:35,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:35,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:18:36,331 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:18:36,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:36,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:37,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:38,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:38,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:39,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:41,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:44,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:44,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:48,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:51,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:52,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:18:52,881 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:18:53,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:53,520 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:18:55,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:59,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:18:59,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:18:59,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:03,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:05,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:05,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:07,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:08,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:19:08,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:19:09,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:09,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:11,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:12,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:12,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:15,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:17,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:18,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:18,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:18,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:18,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:19,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:21,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:22,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:22,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:23,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:23,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:19:23,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:19:23,918 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:19:27,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:29,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:30,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:30,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:33,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:35,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:35,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:35,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:19:38,919 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:19:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:38,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:39,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:40,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:40,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:41,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:42,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:42,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:43,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:45,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:47,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:47,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:49,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:51,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:52,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:52,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:53,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:53,671 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:19:54,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:19:54,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:19:54,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:54,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:55,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:19:56,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:56,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:19:57,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:00,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:01,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:01,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:02,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:04,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:06,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:08,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:09,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:20:09,657 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:20:09,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:09,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:10,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:11,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:11,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:12,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:13,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:14,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:15,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:16,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:18,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:19,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:19,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:20,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:21,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:21,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:22,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:23,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:23,740 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:20:24,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:25,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:20:25,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:20:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:25,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:26,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:27,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:27,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:28,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:28,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:29,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:30,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:30,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:32,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:33,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:33,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:34,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:36,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:40,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:20:40,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:20:42,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:42,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:46,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:48,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:50,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:51,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:51,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:53,807 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:20:55,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:55,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:20:56,203 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:20:56,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:58,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:20:59,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:20:59,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:00,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:00,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:02,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:06,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:09,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:10,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:11,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:11,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:21:11,511 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:21:14,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:16,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:20,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:20,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:20,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:22,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:23,882 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:21:26,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:26,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:21:26,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:21:28,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:28,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:30,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:34,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:34,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:36,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:38,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:38,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:40,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:40,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:42,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:21:42,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:21:44,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:46,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:48,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:49,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:49,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:50,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:52,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:52,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:53,955 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:21:54,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:55,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:56,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:56,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:56,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:56,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:56,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:57,445 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 22:21:57,672 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 22:21:57,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:21:57,733 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 22:21:57,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:21:58,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 22:21:58,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:21:59,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:21:59,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:00,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:02,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:04,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:05,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:05,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:06,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:06,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:08,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:09,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:09,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:10,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:10,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:12,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:12,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:22:12,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:22:14,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:15,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:15,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:16,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:18,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:18,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:18,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:19,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:20,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:22,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:22,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:22,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:23,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:22:24,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:24,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:24,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:25,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:26,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:26,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:27,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:22:28,307 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:22:28,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:28,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:30,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:33,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:33,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:39,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:22:43,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:22:45,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:45,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:46,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:47,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:47,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:48,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:49,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:52,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:52,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:53,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:54,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:22:55,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:22:55,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:55,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:22:58,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:22:58,963 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:22:59,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:00,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:00,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:02,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:03,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:05,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:06,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:06,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:09,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:11,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:12,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:12,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:14,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:23:14,978 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:23:15,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:15,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:15,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:17,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:21,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:23,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:24,240 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:23:25,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:25,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:26,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:29,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:30,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:23:30,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:23:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:30,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:31,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:33,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:35,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:37,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:37,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:38,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:39,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:40,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:41,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:43,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:44,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:44,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:45,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:45,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:23:45,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:23:46,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:46,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:47,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:49,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:51,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:52,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:52,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:53,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:54,314 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:23:54,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:54,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:55,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:56,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:23:57,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:23:59,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:01,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:24:01,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:24:01,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:01,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:02,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:02,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:03,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:05,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:06,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:06,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:07,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:09,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:11,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:11,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:13,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:13,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:14,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:15,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:16,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:24:16,341 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:24:17,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:17,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:17,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:18,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:19,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:20,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:20,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:21,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:22,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:24,386 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:24:24,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:29,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:29,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:24:31,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:24:32,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:35,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:35,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:37,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:38,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:42,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:46,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:46,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:24:46,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:24:47,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:49,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:50,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:52,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:52,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:54,456 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:24:54,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:56,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:56,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:24:59,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:24:59,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:00,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:02,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:25:02,135 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:25:04,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:04,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:05,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:05,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:06,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:10,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:12,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:14,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:14,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:15,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:15,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:17,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:25:17,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:25:18,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:18,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:20,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:20,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:20,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:22,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:23,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:24,531 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:25:26,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:27,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:27,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:28,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:29,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:29,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:30,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:32,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:32,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:25:32,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:25:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:34,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:36,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:38,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:38,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:40,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:41,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:41,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:42,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:43,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:43,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 22:25:44,346 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 22:25:44,568 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 22:25:44,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:44,628 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 22:25:45,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 22:25:45,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:46,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:47,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:25:47,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:25:48,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:50,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:52,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:54,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:54,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:25:56,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:25:58,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:00,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:02,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:03,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:26:03,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:26:04,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:06,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:08,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:10,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:12,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:14,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:16,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:18,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:26:18,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:26:18,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:20,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:22,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:24,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:24,749 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:26:26,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:28,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:30,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:32,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:33,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:26:33,946 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:26:34,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:36,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:38,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:40,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:43,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:45,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:47,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:49,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:26:49,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:26:49,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:51,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:53,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:54,825 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:26:55,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:57,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:26:59,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:01,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:03,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:27:04,574 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:27:05,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:07,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:09,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:11,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:13,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:15,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:17,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:19,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:19,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:27:19,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:27:21,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:23,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:24,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:27:25,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:27,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:29,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:31,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:33,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:35,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:27:35,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:27:35,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:37,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:39,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:41,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:43,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:45,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:47,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:49,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:50,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:27:50,562 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:27:51,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:54,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:54,975 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:27:56,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:27:58,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:00,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:02,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:04,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:06,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:28:06,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:28:06,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:08,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:10,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:12,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:14,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:16,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:18,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:20,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:21,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:28:21,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:28:22,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:24,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:25,048 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:28:26,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:28,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:30,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:32,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:34,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:36,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:28:36,567 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:28:36,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:38,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:40,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:42,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:44,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:46,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:48,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:50,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:51,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:28:51,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:28:52,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:54,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:55,124 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:28:56,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:28:58,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:00,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:02,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:04,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:06,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:07,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:29:07,409 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:29:08,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:10,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:12,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:14,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:16,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:18,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:20,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:22,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:29:22,668 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:29:22,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:24,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:25,202 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:29:26,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:29,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:31,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:33,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:35,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:37,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:29:37,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:29:39,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:41,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:43,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:45,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:47,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:49,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:51,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:29:53,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:29:53,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:55,279 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:29:55,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:57,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:29:59,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:01,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:03,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:05,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:07,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:08,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:30:08,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:30:09,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:11,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:13,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:15,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:17,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:19,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:21,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:23,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:24,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:30:24,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:30:25,354 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:30:25,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:27,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:29,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:31,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:33,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:35,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:37,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:39,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:30:39,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:30:39,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:41,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:43,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:45,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:47,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:49,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:51,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:53,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:55,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:30:55,255 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:30:55,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:30:55,787 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:30:58,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:00,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:02,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:04,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:06,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:08,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:10,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:10,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:31:10,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:31:12,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:14,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:16,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:18,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:20,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:22,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:24,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:25,507 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:31:26,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:31:26,728 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:31:26,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:28,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:30,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:32,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:34,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:36,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:38,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:40,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:42,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:31:42,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:31:42,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:44,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:46,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:48,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:50,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:52,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:54,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:55,580 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:31:56,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:31:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:31:57,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:31:58,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:00,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:02,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:04,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:06,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:08,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:10,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:12,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:13,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:32:13,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:32:14,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:16,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:18,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:20,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:23,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:25,655 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:32:25,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:27,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:28,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:32:28,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:32:29,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:31,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:33,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:35,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:37,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:39,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:41,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:43,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:43,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:32:43,993 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:32:45,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:47,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:49,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:51,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:53,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:55,729 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:32:55,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:57,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:32:59,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:32:59,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:32:59,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:01,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:03,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:05,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:07,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:09,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:11,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:13,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:14,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:33:14,960 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:33:15,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:17,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:19,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:21,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:23,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:25,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:33:25,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:27,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:29,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:30,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:33:30,361 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:33:31,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:33,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:35,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:37,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:39,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:41,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:43,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:45,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:33:45,721 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:33:45,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:47,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:50,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:52,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:54,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:55,876 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:33:56,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:33:58,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:00,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:01,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:34:01,295 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:34:02,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:04,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:06,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:08,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:10,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:12,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:14,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:16,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:34:16,631 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:34:16,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:18,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:20,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:22,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:24,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:25,950 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:34:26,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:28,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:30,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:32,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:34:32,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:34:32,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:34,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:36,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:38,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:40,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:42,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:44,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:46,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:34:47,866 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:34:48,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:50,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:52,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:54,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:56,024 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:34:56,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:34:58,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:00,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:02,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:03,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:35:03,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:35:04,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:06,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:08,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:10,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:12,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:14,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:16,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:18,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:35:18,568 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:35:18,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:20,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:22,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:24,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:26,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:35:26,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:28,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:30,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:32,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:33,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:35:33,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:35:34,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:37,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:39,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:41,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:43,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:45,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:47,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:49,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:35:49,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:35:49,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:51,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:53,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:55,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:56,152 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:35:57,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:35:59,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:01,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:03,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:04,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:36:04,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:36:05,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:07,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:09,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:11,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:13,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:15,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:17,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:19,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:19,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:36:19,913 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:36:21,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:23,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:25,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:26,226 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:36:27,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:29,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:31,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:33,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:35,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:36:35,342 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:36:35,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:37,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:39,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:41,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:43,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:45,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:47,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:49,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:50,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:36:50,628 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:36:51,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:53,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:55,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:56,303 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:36:57,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:36:59,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:01,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:03,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:05,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:37:05,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:37:05,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:07,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:09,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:11,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:13,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:15,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:17,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:19,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:21,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:37:21,183 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:37:21,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:24,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:26,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:37:26,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:28,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:30,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:32,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:34,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:36,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:37:36,466 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:37:36,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:38,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:40,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:42,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:44,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:46,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:48,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:50,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:51,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:37:51,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:37:52,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:54,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:56,456 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:37:56,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:37:58,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:00,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:02,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:04,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:06,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:07,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:38:07,006 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:38:08,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:10,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:12,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:14,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:16,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:18,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:20,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:22,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:38:22,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:38:22,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:24,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:26,532 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:38:26,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:28,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:30,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:32,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:34,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:36,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:38:37,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:38:38,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:40,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:42,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:44,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:47,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:49,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:51,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:53,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:38:53,096 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:38:53,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:55,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:56,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:38:57,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:38:59,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:01,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:03,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:05,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:07,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:08,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:39:08,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:39:09,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:11,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:13,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:15,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:17,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:19,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:21,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:23,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:39:23,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:39:23,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:25,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:26,661 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:39:27,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:29,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:31,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:33,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:35,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:37,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:39,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:39:39,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:39:39,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:41,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:43,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:45,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:47,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:49,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:51,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:53,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:39:54,565 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:39:55,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:56,721 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:39:57,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:39:59,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:01,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:03,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:05,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:07,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:40:09,866 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:40:09,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:11,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:14,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:16,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:19,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:21,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:23,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:25,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:25,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:40:25,102 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:40:26,797 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:40:27,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:29,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:31,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:33,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:35,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:37,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:39,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:40,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:40:40,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:40:41,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:43,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:45,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:47,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:49,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:51,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:53,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:55,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:56,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:40:56,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:40:56,869 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:40:57,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:40:59,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:01,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:03,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:05,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:07,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:09,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:11,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:11,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:41:11,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:41:13,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:15,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:17,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:19,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:21,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:23,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:25,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:26,944 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:41:26,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:41:26,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:41:27,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:29,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:31,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:33,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:35,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:37,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:39,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:41,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:42,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:41:42,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:41:43,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:45,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:48,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:50,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:52,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:54,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:56,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:41:57,020 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:41:57,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:41:57,595 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:41:58,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:00,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:02,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:04,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:06,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:08,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:10,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:12,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:12,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:42:12,837 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:42:14,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:16,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:18,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:20,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:22,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:24,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:26,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:27,092 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:42:28,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:42:28,144 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:42:30,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:32,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:34,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:36,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:38,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:40,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:42,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:43,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:42:43,502 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:42:44,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:46,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:48,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:50,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:52,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:54,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:56,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:57,166 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:42:58,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:42:58,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:42:58,909 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:43:00,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:02,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:04,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:06,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:08,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:10,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:12,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:14,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:14,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:43:14,203 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:43:16,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:18,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:20,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:22,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:24,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:26,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:27,244 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:43:28,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:29,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:43:29,540 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:43:30,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:32,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:34,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:36,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:38,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:40,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:42,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:44,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:44,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:43:44,777 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:43:46,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:48,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:50,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:52,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:55,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:57,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:43:57,318 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:43:59,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:00,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:44:00,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:44:01,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:03,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:05,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:07,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:09,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:11,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:13,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:15,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:15,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:44:15,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:44:17,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:19,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:21,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:23,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:25,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:27,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:27,393 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:44:29,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:30,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:44:30,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:44:31,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:33,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:35,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:37,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:39,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:41,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:43,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:45,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:44:46,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:44:47,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:49,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:51,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:53,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:55,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:44:57,466 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:44:58,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:00,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:01,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:45:01,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:45:02,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:04,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:06,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:08,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:10,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:12,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:14,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:16,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:16,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:45:16,742 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:45:18,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:20,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:22,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:24,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:26,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:27,539 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:45:28,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:30,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:32,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:45:32,109 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:45:32,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:34,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:36,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:38,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:40,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:42,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:44,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:46,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:47,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:45:47,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:45:48,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:50,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:52,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:54,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:56,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:45:57,612 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:45:58,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:00,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:02,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:02,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:46:02,820 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:46:04,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:06,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:08,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:10,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:12,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:14,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:16,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:18,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:46:18,253 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:46:20,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:22,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:24,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:26,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:27,694 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:46:28,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:30,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:32,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:33,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:46:33,548 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:46:34,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:37,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:39,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:41,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:43,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:45,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:47,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:48,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:46:48,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:46:49,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:51,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:53,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:55,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:57,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:46:57,770 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:46:59,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:01,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:03,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:04,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:47:04,337 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:47:05,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:07,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:09,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:11,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:13,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:15,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:17,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:19,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:20,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:47:20,257 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:47:21,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:23,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:25,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:27,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:27,843 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:47:29,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:31,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:33,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:35,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:35,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:47:35,838 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:47:37,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:39,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:41,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:43,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:45,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:47,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:49,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:51,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:47:51,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:47:53,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:55,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:57,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:47:57,918 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:47:59,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:01,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:03,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:05,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:06,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:48:06,859 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:48:07,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:09,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:11,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:14,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:16,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:18,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:20,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:22,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:22,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:48:22,216 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:48:24,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:26,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:27,994 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:48:28,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:30,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:32,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:34,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:36,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:37,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:48:37,479 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:48:38,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:40,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:42,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:44,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:46,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:48,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:50,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:52,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:52,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:48:52,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:48:54,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:56,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:48:58,070 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:48:58,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:00,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:02,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:04,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:06,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:08,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:49:08,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:49:08,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:10,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:12,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:14,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:16,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:18,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:20,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:22,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:23,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:49:23,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:49:24,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:27,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:28,143 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:49:29,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:31,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:33,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:35,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:37,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:38,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:49:38,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:49:39,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:41,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:43,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:45,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:47,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:49,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:51,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:53,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:49:54,302 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:49:55,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:57,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:49:58,217 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:49:59,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:01,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:03,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:05,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:07,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:09,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:09,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:50:09,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:50:11,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:13,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:15,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:17,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:19,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:21,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:23,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:24,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:50:24,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:50:25,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:27,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:28,294 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:50:29,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:31,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:33,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:35,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:37,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:39,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:40,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:50:40,376 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:50:41,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:44,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:46,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:48,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:50,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:52,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:54,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:55,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:50:55,842 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:50:56,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:58,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:50:58,372 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:51:00,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:02,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:04,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:06,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:08,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:10,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:11,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:51:11,139 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:51:12,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:14,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:16,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:18,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:20,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:22,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:24,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:26,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:51:26,982 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:51:28,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:28,442 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:51:30,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:32,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:34,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:36,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:38,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:40,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:42,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:51:42,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:51:44,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:46,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:48,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:50,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:52,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:54,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:56,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:57,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:51:57,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:51:58,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:51:58,508 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:52:00,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:02,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:04,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:07,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:09,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:11,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:13,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:52:13,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:52:13,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:15,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:17,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:19,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:21,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:23,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:25,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:27,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:28,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:52:28,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:52:28,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:52:29,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:31,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:33,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:35,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:37,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:39,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:41,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:43,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:43,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:52:43,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:52:45,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:47,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:49,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:51,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:53,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:55,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:57,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:58,657 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:52:59,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:52:59,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:52:59,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:53:01,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:03,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:05,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:07,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:10,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:12,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:14,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:14,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:53:14,653 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:53:16,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:18,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:20,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:22,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:24,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:26,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:28,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:28,731 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:53:30,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:53:30,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:53:30,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:32,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:34,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:36,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:38,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:40,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:42,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:44,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:45,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:53:45,336 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:53:46,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:48,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:50,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:52,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:54,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:56,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:58,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:53:58,808 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:54:00,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:00,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:54:00,696 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:54:02,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:04,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:06,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:08,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:10,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:12,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:14,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:16,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:54:16,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:54:16,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:18,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:20,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:23,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:25,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:27,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:28,881 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:54:29,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:31,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:31,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:54:31,469 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:54:33,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:35,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:37,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:39,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:41,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:43,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:45,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:46,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:54:46,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:54:47,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:49,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:51,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:53,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:55,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:57,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:54:58,954 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:54:59,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:01,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:55:02,513 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:55:03,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:05,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:07,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:09,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:11,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:13,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:15,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:17,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:17,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:55:17,806 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:55:19,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:21,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:23,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:25,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:27,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:29,030 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:55:29,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:31,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:33,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:55:33,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:55:33,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:35,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:37,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:39,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:41,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:43,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:45,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:47,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:48,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:55:48,636 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:55:49,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:51,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:53,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:55,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:57,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:55:59,103 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:55:59,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:01,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:04,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:56:04,054 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:56:04,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:06,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:08,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:10,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:12,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:14,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:16,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:18,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:19,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:56:19,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:56:20,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:22,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:24,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:26,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:28,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:29,168 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:56:30,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:32,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:34,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:34,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:56:34,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:56:36,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:38,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:40,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:42,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:44,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:46,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:48,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:56:50,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:56:50,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:52,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:54,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:56,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:58,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:56:59,230 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:57:00,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:02,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:04,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:05,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:57:05,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:57:06,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:08,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:10,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:12,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:14,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:16,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:18,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:20,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:20,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:57:20,686 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:57:22,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:24,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:26,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:28,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:29,302 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:57:30,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:32,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:34,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:36,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:57:36,073 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:57:37,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:39,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:41,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:43,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:45,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:47,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:49,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:51,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:51,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:57:51,573 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:57:53,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:55,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:57,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:57:59,378 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:57:59,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:01,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:03,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:05,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:06,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:58:06,903 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:58:07,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:09,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:11,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:13,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:15,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:17,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:19,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:21,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:22,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:58:22,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:58:23,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:25,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:27,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:29,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:29,454 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:58:31,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:33,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:35,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:37,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:58:37,456 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:58:39,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:41,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:43,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:45,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:47,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:49,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:51,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:52,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:58:52,862 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:58:53,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:55,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:57,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:59,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:58:59,529 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:59:01,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:03,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:05,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:07,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:08,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:59:08,123 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:59:09,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:11,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:13,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:15,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:17,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:19,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:22,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:23,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:59:23,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:59:24,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:26,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:28,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:29,609 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 22:59:30,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:32,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:34,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:36,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:38,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:38,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:59:38,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:59:40,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:42,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:44,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:46,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:48,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:50,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:52,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:54,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 22:59:54,004 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 22:59:54,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:56,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:58,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 22:59:59,685 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:00:00,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:02,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:04,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:06,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:08,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:09,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:00:09,241 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:00:10,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:12,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:14,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:16,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:18,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:20,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:22,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:24,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:24,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:00:24,516 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:00:26,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:28,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:29,758 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:00:30,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:32,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:34,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:36,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:38,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:40,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:00:40,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:00:40,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:42,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:45,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:47,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:49,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:51,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:53,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:55,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:55,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:00:55,676 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:00:57,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:59,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:00:59,833 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:01:01,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:03,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:05,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:07,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:09,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:11,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:01:12,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:01:13,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:15,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:17,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:19,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:21,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:23,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:25,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:27,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:28,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:01:28,281 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:01:29,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:29,911 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:01:31,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:33,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:35,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:37,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:39,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:41,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:43,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:43,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:01:43,807 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:01:45,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:47,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:49,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:51,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:53,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:55,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:58,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:01:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:01:59,158 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:01:59,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:02:00,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:02,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:04,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:06,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:08,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:10,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:12,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:14,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:02:14,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:02:14,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:16,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:18,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:20,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:22,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:24,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:26,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:28,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:29,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:02:29,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:02:30,144 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:02:30,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:32,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:34,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:36,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:38,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:40,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:42,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:44,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:45,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:02:45,236 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:02:46,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:48,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:50,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:52,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:54,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:56,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:02:58,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:00,132 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:03:00,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:01,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:03:01,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:03:02,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:04,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:06,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:08,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:10,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:12,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:14,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:16,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:03:16,591 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:03:17,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:19,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:21,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:23,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:25,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:27,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:29,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:30,207 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:03:31,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:31,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:03:31,915 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:03:33,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:35,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:37,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:39,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:41,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:43,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:45,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:47,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:03:47,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:03:47,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:49,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:51,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:53,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:55,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:57,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:03:59,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:00,281 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:04:01,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:02,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:04:02,742 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:04:03,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:05,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:07,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:09,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:11,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:13,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:15,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:17,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:18,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:04:18,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:04:19,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:21,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:23,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:25,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:27,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:29,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:30,355 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:04:31,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:33,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:04:33,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:04:34,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:36,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:38,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:40,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:42,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:44,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:46,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:48,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:04:48,539 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:04:48,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:50,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:52,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:54,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:56,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:04:58,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:00,428 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:05:00,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:02,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:03,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:05:03,772 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:05:04,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:06,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:08,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:10,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:12,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:14,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:16,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:18,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:19,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:05:19,041 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:05:20,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:22,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:24,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:26,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:28,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:30,504 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:05:30,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:32,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:34,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:05:34,276 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:05:34,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:36,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:38,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:40,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:42,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:44,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:46,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:48,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:49,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:05:49,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:05:50,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:52,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:54,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:56,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:05:58,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:00,576 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:06:00,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:02,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:04,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:05,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:06:05,000 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:06:06,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:08,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:10,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:12,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:14,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:17,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:19,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:20,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:06:20,220 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:06:21,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:23,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:25,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:27,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:29,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:30,650 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:06:31,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:33,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:35,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:06:35,448 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:06:35,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:37,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:39,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:41,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:43,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:45,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:47,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:49,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:50,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:06:50,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:06:51,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:53,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:55,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:57,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:06:59,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:00,726 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:07:01,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:03,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:05,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:06,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:07:06,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:07:07,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:09,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:11,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:13,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:15,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:17,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:19,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:21,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:07:21,345 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:07:21,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:23,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:25,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:27,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:29,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:30,801 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:07:31,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:33,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:35,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:36,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:07:36,613 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:07:37,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:07:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:07:51,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:08:00,878 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:08:07,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:08:07,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:08:22,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:08:22,380 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:08:30,953 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:08:35,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:08:35,245 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 23:08:35,296 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 23:08:35,372 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 23:08:35,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 23:08:35,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:08:36,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:08:37,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:08:37,640 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:08:41,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:08:52,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:08:52,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:08:57,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:09:01,044 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:09:08,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:09:08,284 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:09:23,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:09:23,515 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:09:31,127 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:09:38,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:09:38,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:09:53,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:09:53,976 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:10:01,213 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:10:09,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:10:09,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:10:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:10:24,625 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:10:31,296 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:10:40,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:10:40,027 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:10:55,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:10:55,377 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:11:01,382 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:11:12,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:11:12,269 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:11:27,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:11:27,632 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:11:31,464 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:11:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:11:43,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:11:58,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:11:58,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:12:01,546 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:12:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:12:14,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:12:30,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:12:30,157 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:12:32,439 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:12:47,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:12:47,441 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:13:01,703 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:13:02,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:03,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:13:03,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:13:04,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:05,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:06,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:08,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:10,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:10,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:12,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:12,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:14,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:16,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:16,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:17,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:17,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:18,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:13:18,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:13:18,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:19,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:19,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:20,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:22,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:22,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:24,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:25,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:25,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:26,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:28,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:28,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:30,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:30,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,767 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:13:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:31,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:32,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:13:33,711 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:13:33,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:33,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:34,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:36,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:41,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:42,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:45,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:48,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:49,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:13:49,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:13:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:50,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:52,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:13:54,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:54,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:13:56,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:00,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:01,837 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:14:03,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:04,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:14:04,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:14:07,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:10,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:11,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:11,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:12,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:12,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:12,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:12,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:13,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:17,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:17,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:17,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:19,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:19,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:14:19,893 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:14:21,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:21,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:22,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:23,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:24,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:25,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:29,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:30,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:31,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:31,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:14:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:33,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:35,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:14:35,303 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:14:35,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:38,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:38,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:39,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:39,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:41,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:43,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:45,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:47,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:49,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:50,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:14:50,629 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:14:51,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:52,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:53,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:54,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:54,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:14:55,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:57,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:14:59,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:00,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:01,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:01,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:15:03,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:03,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:03,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:15:05,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:15:05,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:06,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:06,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:07,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:09,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:11,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:11,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:13,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:14,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:14,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:15,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:16,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:16,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:17,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:18,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:18,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:19,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:21,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:15:21,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:15:21,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:22,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:22,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:23,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:25,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:26,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:26,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:27,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:29,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:31,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:32,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:15:35,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:35,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:36,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:15:36,522 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:15:37,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:37,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:38,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:39,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:41,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:41,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:44,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:45,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:49,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:51,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:15:51,831 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:15:53,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:54,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:57,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:15:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:58,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:15:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:01,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:02,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:16:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:05,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:06,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:07,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:16:07,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:16:08,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:10,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:10,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:11,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:12,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:14,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:14,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:16,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:16,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:18,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:20,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:21,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:21,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:22,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:16:22,293 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:16:23,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:23,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:24,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:26,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:26,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:27,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:27,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:28,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:30,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:32,192 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:16:32,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:34,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:35,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:35,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:36,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:37,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:37,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:16:37,518 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:16:39,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:40,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:42,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:42,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:44,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:46,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:48,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:50,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:51,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:51,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:52,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:52,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:16:52,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:16:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:53,717 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 23:16:53,927 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 23:16:54,005 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 23:16:54,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:54,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:55,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 23:16:55,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:56,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:56,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:16:57,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:57,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:16:59,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:00,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:00,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:01,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:02,325 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:17:03,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:05,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:05,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:07,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:07,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:17:08,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:17:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:08,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:09,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:11,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:11,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:11,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:12,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:13,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:14,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:15,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:16,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:17,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:18,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:18,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:19,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:20,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:21,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:23,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:23,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:23,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:17:23,581 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:17:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:24,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:25,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:25,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:27,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:32,398 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:17:37,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:37,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:37,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:38,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:17:38,807 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:17:39,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:41,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:41,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:42,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:46,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:47,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:50,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:54,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:54,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:17:54,039 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:17:58,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:17:59,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:17:59,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:02,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:02,472 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:18:04,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:05,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:05,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:07,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:08,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:08,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:09,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:18:09,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:18:12,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:13,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:14,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:16,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:20,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:20,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:21,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:22,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:23,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:23,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:25,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:18:25,086 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:18:26,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:26,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:26,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:27,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:28,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:28,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:30,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:32,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:32,542 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:18:36,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:36,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:36,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:38,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:39,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:39,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:40,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:40,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:18:40,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:18:41,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:41,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:42,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:43,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:44,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:46,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:48,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:48,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:48,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:50,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:52,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:54,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:54,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:54,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:55,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:18:55,816 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:18:56,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:56,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:56,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:57,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:58,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:18:59,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:18:59,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:00,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:02,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:02,617 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:19:04,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:04,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:04,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:05,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:06,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:07,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:08,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:10,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:10,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:10,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:11,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:19:11,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:19:12,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:12,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:12,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:13,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:14,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:14,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:15,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:16,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:16,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:17,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:18,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:20,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:20,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:20,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:21,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:22,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:24,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:26,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:19:26,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:19:27,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:27,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:30,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:32,686 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:19:34,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:38,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:41,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:41,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:19:41,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:19:42,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:42,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:47,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:47,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:48,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:51,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:55,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:19:55,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:55,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:19:57,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:19:57,240 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:19:59,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:01,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:02,756 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:20:05,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:08,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:08,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:09,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:09,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:10,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:11,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:12,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:20:12,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:20:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:15,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:15,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:16,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:17,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:19,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:20,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:20,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:21,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:23,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:23,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:25,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:26,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:26,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:27,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:27,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:20:27,707 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:20:30,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:30,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:31,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:31,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:32,826 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:20:33,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:33,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:33,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:35,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:36,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:37,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:38,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:38,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:39,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:39,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:41,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:41,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:42,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:20:42,936 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:20:43,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:44,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:45,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:46,529 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 23:20:46,762 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 23:20:46,822 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 23:20:47,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 23:20:47,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:49,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:49,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:49,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:51,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:53,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:55,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:55,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:55,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:57,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:20:58,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:20:58,409 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:20:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:58,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:20:59,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:01,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:02,961 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:21:03,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:03,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:03,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:05,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:06,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:06,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:07,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:07,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:08,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:09,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:10,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:10,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:11,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:13,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:13,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:21:13,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:21:13,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:13,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:14,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:15,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:16,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:17,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:17,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:18,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:19,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:22,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:25,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:28,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:21:28,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:21:32,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:33,038 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:21:36,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:36,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:38,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:41,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:41,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:42,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:44,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:21:44,211 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:21:46,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:48,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:48,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:50,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:52,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:53,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:54,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:55,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:21:56,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:21:59,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:21:59,477 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:22:00,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:02,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:03,108 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:22:04,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:04,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:06,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:08,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:11,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:12,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:13,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:13,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:14,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:22:14,854 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:22:15,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:15,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:16,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:19,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:19,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:20,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:22,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:22,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:24,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:24,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:24,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:26,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:26,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:28,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:29,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:29,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:30,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:22:30,191 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:22:30,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:30,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:31,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:33,169 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:22:34,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:34,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:34,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:36,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:37,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:37,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:38,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:38,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:39,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:40,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:41,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:41,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:41,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:41,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:42,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:42,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:44,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:45,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:22:45,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:22:46,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:46,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:48,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:50,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:52,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:54,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:54,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:54,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:56,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:57,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:57,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:22:58,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:22:58,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:00,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:00,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:23:00,686 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:23:02,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:02,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:03,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:23:04,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:04,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:04,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:06,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:07,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:08,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:08,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:10,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:11,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:12,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:12,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:13,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:14,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:14,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:15,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:16,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:23:16,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:23:17,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:27,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:29,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:29,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:31,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:31,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:23:31,655 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:23:33,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:23:37,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:37,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:39,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:43,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:46,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:23:46,884 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:23:47,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:51,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:52,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:53,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:55,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:55,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:56,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:57,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:23:58,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:58,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:23:59,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:01,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:02,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:24:02,187 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:24:03,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:03,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:24:05,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:05,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:05,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:09,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:11,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:13,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:13,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:14,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:17,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:17,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:24:17,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:24:19,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:19,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:19,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:21,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:22,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:23,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:24,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:25,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:25,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:28,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:29,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:30,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:31,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:32,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:32,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:24:32,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:24:33,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:33,458 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:24:35,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:35,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:36,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:37,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:37,733 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 23:24:37,954 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 23:24:38,018 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 23:24:38,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 23:24:38,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:40,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:40,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:41,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:42,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:43,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:44,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:46,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:47,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:47,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:24:47,991 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:24:48,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:48,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:48,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:49,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:50,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:52,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:52,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:53,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:53,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:54,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:54,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:56,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:58,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:24:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:24:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:00,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:00,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:01,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:02,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:03,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:25:03,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:25:03,576 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:25:03,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:03,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:04,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:04,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:06,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:07,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:07,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:08,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:09,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:10,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:12,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:16,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:16,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:18,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:25:18,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:25:22,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:26,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:26,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:28,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:30,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:33,647 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:25:33,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:25:33,789 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:25:34,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:36,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:36,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:38,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:42,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:46,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:48,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:48,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:48,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:49,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:25:49,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:25:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:52,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:54,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:25:56,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:56,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:57,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:25:58,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:00,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:00,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:00,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:03,717 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:26:04,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:04,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:26:04,476 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:26:05,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:05,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:06,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:08,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:12,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:14,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:16,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:16,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:18,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:19,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:19,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:26:19,769 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:26:20,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:20,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:20,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:21,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:22,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:23,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:23,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:24,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:24,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:26,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:27,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:28,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:28,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:31,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:32,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:32,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:32,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:32,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:33,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:33,790 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:26:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:34,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:26:34,997 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:26:35,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:35,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:35,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:37,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:38,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:38,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:39,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:40,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:41,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:43,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:45,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:46,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:47,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:48,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:49,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:26:50,328 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:26:51,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:51,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:51,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:52,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:53,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:55,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:57,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:58,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:58,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:26:59,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:26:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:01,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:03,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:03,862 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:27:04,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:04,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:05,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:05,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:27:05,569 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:27:07,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:09,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:09,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:09,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:13,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:17,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:20,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:27:20,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:27:21,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:23,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:23,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:25,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:29,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:29,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:29,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:31,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:33,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:33,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:27:36,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:27:36,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:27:37,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:37,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:37,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:40,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:41,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:43,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:43,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:47,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:47,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:48,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:49,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:51,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:27:51,270 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:27:53,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:53,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:53,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:27:55,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:55,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:58,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:27:59,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:01,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:01,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:01,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:03,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:04,006 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:28:06,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:28:06,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:28:07,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:09,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:11,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:13,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:13,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:15,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:15,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:15,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:17,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:19,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:21,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:21,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:28:21,841 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:28:22,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:22,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:23,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:24,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:24,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:25,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:26,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:27,326 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 23:28:27,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:27,525 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 23:28:27,595 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 23:28:28,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 23:28:28,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:29,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:29,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:29,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:30,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:31,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:31,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:32,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:33,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:34,122 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:28:35,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:35,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:36,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:28:37,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:28:37,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:37,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:37,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:39,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:41,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:41,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:43,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:45,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:45,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:45,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:47,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:47,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:48,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:49,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:51,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:52,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:28:52,345 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:28:52,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:52,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:53,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:54,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:54,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:55,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:57,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:28:58,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:58,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:28:59,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:01,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:02,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:02,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:04,191 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:29:06,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:06,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:07,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:29:07,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:29:09,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:09,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:09,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:11,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:12,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:15,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:20,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:23,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:29:23,436 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:29:24,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:24,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:28,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:30,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:30,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:34,269 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:29:34,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:34,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:36,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:37,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:37,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:38,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:29:38,791 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:29:39,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:39,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:40,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:42,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:42,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:46,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:46,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:46,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:48,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:50,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:50,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:52,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:54,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:29:54,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:29:54,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:54,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:55,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:55,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:56,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:29:57,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:57,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:29:58,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:02,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:03,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:03,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:04,341 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:30:04,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:06,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:08,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:09,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:30:09,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:30:10,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:10,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:12,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:14,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:15,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:16,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:17,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:17,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:18,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:18,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:19,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:20,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:21,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:21,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:22,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:24,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:30:24,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:30:24,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:24,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:26,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:27,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:27,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:28,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:30,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:30,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:31,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:32,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:32,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,422 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:30:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:34,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:35,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:35,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:36,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:38,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:39,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:39,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:30:39,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:30:40,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:40,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:40,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:42,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:42,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:43,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:44,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:46,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:46,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:48,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:48,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:30:50,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:52,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:30:55,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:30:55,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:31:02,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:02,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:02,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:04,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:31:08,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:10,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:31:10,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:31:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:11,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:13,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:15,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:17,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:18,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:18,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:19,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:22,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:22,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:23,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:26,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:31:26,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:31:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:26,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:27,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:28,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:29,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:33,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:33,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:34,552 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:31:35,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:36,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:38,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:39,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:41,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:41,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:31:41,474 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:31:41,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:45,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:46,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:46,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:47,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:49,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:52,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:52,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:53,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:53,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:55,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:56,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:56,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:31:56,790 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:31:57,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:31:58,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:58,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:31:59,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:00,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:00,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:01,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:03,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:04,621 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:32:07,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:09,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:10,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:10,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:11,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:12,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:32:12,048 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:32:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-23 23:32:12,986 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-23 23:32:13,201 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-23 23:32:13,262 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-23 23:32:13,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-23 23:32:13,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:15,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:17,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:19,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:21,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:23,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:25,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:27,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:32:27,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:32:27,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:29,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:31,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:33,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:34,714 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:32:35,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:37,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:39,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:41,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:42,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:32:42,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:32:43,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:45,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:47,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:49,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:51,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:53,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:56,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:32:58,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:32:58,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:32:58,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:00,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:02,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:04,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:04,787 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:33:06,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:08,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:10,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:12,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:13,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:33:13,444 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:33:14,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:16,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:18,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:20,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:22,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:24,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:26,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:28,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:28,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:33:28,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:33:30,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:32,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:34,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:34,855 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:33:36,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:38,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:40,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:42,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:43,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:33:43,983 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:33:44,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:46,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:48,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:50,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:52,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:54,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:56,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:58,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:33:59,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:33:59,702 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:34:00,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:02,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:04,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:04,932 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:34:06,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:08,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:10,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:12,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:14,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:14,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:34:14,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:34:16,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:18,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:21,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:23,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:25,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:27,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:29,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:30,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:34:30,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:34:31,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:33,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:35,005 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:34:35,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:37,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:39,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:41,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:43,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:45,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:34:45,750 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:34:47,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:49,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:51,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:53,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:55,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:57,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:34:59,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:00,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:35:00,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:35:01,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:03,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:05,081 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:35:05,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:07,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:09,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:11,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:13,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:15,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:16,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:35:16,306 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:35:17,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:19,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:21,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:23,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:25,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:27,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:29,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:35:31,557 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:35:31,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:33,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:35,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:35:35,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:37,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:39,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:41,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:43,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:45,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:46,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:35:46,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:35:47,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:49,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:51,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:53,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:56,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:35:58,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:00,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:02,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:36:02,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:36:02,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:04,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:05,235 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:36:06,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:08,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:10,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:12,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:14,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:16,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:17,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:36:17,551 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:36:18,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:20,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:22,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:24,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:26,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:28,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:30,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:32,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:32,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:36:32,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:36:34,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:35,319 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:36:36,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:38,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:40,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:42,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:44,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:46,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:48,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:36:48,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:36:48,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:50,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:52,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:54,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:56,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:36:58,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:00,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:02,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:03,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:37:03,418 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:37:04,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:05,394 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:37:06,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:08,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:10,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:12,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:14,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:16,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:18,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:18,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:37:18,938 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:37:20,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:22,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:24,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:27,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:29,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:31,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:33,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:34,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:37:34,251 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:37:35,471 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:37:35,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:37,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:39,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:41,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:43,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:45,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:47,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:49,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:37:49,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:37:49,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:51,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:53,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:55,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:57,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:37:59,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:01,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:03,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:04,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:38:04,782 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:38:05,547 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:38:05,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:07,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:09,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:11,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:13,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:15,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:17,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:19,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:20,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:38:20,077 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:38:21,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:23,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:25,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:27,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:29,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:31,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:33,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:38:35,461 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:38:35,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:35,693 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:38:37,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:39,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:41,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:43,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:45,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:47,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:50,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:50,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:38:50,695 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:38:52,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:54,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:56,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:38:58,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:00,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:02,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:04,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:05,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:39:06,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:39:06,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:39:06,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:08,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:10,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:12,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:14,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:16,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:18,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:20,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:21,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:39:21,358 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:39:22,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:24,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:26,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:28,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:30,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:32,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:34,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:35,775 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:39:36,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:39:36,661 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:39:36,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:38,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:40,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:42,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:44,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:46,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:48,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:50,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:52,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:39:52,268 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:39:52,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:54,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:56,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:39:58,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:00,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:02,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:04,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:05,851 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:40:06,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:07,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:40:07,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:40:08,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:10,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:12,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:14,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:16,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:18,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:21,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:22,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:40:22,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:40:23,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:25,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:27,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:29,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:31,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:33,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:35,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:35,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:40:37,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:38,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:40:38,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:40:39,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:41,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:43,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:45,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:47,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:49,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:51,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:53,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:40:53,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:40:53,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:55,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:57,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:40:59,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:01,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:03,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:05,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:06,002 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:41:07,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:08,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:41:08,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:41:09,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:11,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:13,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:15,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:17,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:19,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:21,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:23,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:24,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:41:24,003 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:41:25,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:27,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:29,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:31,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:33,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:35,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:36,076 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:41:37,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:39,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:41:39,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:41:39,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:41,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:43,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:45,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:47,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:49,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:51,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:53,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:54,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:41:54,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:41:55,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:57,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:41:59,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:01,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:03,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:05,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:06,150 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:42:07,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:09,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:09,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:42:09,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:42:11,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:13,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:15,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:18,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:20,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:22,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:24,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:25,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:42:25,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:42:26,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:28,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:30,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:32,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:34,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:36,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:42:36,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:38,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:40,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:42:40,296 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:42:40,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:42,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:44,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:46,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:48,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:50,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:52,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:54,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:55,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:42:55,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:42:56,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:42:58,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:00,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:02,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:04,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:06,298 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:43:06,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:08,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:10,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:11,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:43:11,069 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:43:12,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:14,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:16,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:18,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:20,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:22,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:24,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:26,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:43:26,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:43:26,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:28,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:30,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:32,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:34,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:36,368 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:43:36,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:38,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:40,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:41,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:43:41,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:43:42,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:44,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:46,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:48,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:50,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:52,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:54,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:56,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:43:56,985 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:43:57,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:43:59,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:01,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:03,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:05,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:06,441 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:44:07,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:09,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:11,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:12,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:44:12,217 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:44:13,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:15,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:17,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:19,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:21,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:23,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:25,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:27,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:44:27,524 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:44:27,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:29,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:31,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:33,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:35,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:36,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:44:37,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:39,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:41,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:44:42,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:44:43,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:45,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:47,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:49,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:51,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:53,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:55,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:57,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:44:58,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:44:58,171 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:44:59,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:01,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:03,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:05,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:06,594 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:45:07,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:09,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:11,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:13,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:45:13,525 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:45:14,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:16,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:18,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:20,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:22,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:24,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:26,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:28,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:45:28,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:45:28,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:30,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:32,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:34,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:36,670 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:45:36,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:38,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:40,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:42,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:44,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:45:44,151 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:45:44,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:46,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:48,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:50,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:52,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:54,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:56,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:58,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:45:59,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:45:59,546 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:46:00,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:02,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:04,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:06,743 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:46:06,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:08,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:10,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:12,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:14,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:46:14,864 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:46:17,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:19,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:21,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:23,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:25,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:27,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:29,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:30,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:46:30,180 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:46:31,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:33,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:35,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:36,820 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:46:37,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:39,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:41,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:43,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:45,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:46:45,412 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:46:45,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:47,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:49,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:51,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:53,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:55,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:57,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:46:59,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:00,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:47:00,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:47:01,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:03,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:05,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:06,897 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:47:07,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:09,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:11,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:13,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:15,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:16,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:47:16,265 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:47:17,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:19,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:21,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:23,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:25,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:27,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:29,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:31,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:47:31,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:47:31,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:33,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:35,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:36,968 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:47:38,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:40,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:42,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:44,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:46,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:47:46,887 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:47:46,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:48,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:50,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:52,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:54,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:56,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:47:58,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:00,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:02,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:48:02,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:48:02,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:04,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:06,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:07,043 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:48:08,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:10,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:12,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:14,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:16,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:17,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:48:17,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:48:18,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:20,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:22,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:24,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:26,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:28,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:30,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:32,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:48:32,783 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:48:32,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:34,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:36,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:37,118 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:48:38,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:40,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:42,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:44,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:46,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:48:48,195 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:48:48,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:50,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:52,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:54,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:56,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:48:58,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:00,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:02,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:03,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:49:03,635 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:49:04,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:06,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:07,178 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:49:08,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:10,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:12,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:14,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:16,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:18,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:19,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:49:19,014 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:49:20,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:22,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:24,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:26,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:28,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:30,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:33,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:49:34,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:49:35,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:37,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:49:37,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:39,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:41,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:43,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:45,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:47,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:49,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:49:49,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:49:49,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:51,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:53,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:55,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:57,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:49:59,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:01,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:03,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:50:04,929 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:50:05,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:07,323 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:50:07,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:09,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:11,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:13,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:15,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:17,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:19,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:20,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:50:20,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:50:21,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:23,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:25,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:27,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:29,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:31,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:33,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:50:35,458 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:50:35,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:37,398 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:50:37,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:39,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:41,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:43,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:45,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:47,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:49,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:50,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:50:50,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:50:51,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:53,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:55,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:57,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:50:59,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:02,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:04,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:06,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:51:06,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:51:06,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:07,468 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:51:08,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:10,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:12,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:14,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:16,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:18,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:20,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:21,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:51:21,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:51:22,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:24,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:26,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:28,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:30,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:32,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:34,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:36,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:51:36,957 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:51:36,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:37,543 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:51:38,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:41,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:43,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:45,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:47,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:49,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:51,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:52,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:51:52,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:51:53,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:55,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:57,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:51:59,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:01,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:03,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:05,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:07,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:07,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:52:07,516 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:52:07,802 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:52:09,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:11,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:13,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:15,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:17,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:19,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:21,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:22,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:52:22,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:52:23,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:25,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:27,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:29,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:31,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:33,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:35,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:37,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:37,701 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:52:38,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:52:38,181 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:52:39,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:41,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:44,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:46,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:48,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:50,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:52,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:53,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:52:53,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:52:54,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:56,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:52:58,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:00,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:02,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:04,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:06,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:07,775 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:53:08,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:08,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:53:08,732 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:53:10,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:12,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:14,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:16,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:18,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:20,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:22,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:24,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:24,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:53:24,148 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:53:26,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:28,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:30,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:32,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:34,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:36,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:37,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:53:38,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:39,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:53:39,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:53:40,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:42,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:44,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:46,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:48,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:50,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:52,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:54,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:53:54,689 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:53:56,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:53:58,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:00,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:02,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:04,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:06,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:07,928 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:54:09,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:10,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:54:10,080 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:54:11,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:13,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:15,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:17,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:19,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:21,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:23,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:25,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:25,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:54:25,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:54:27,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:29,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:31,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:33,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:35,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:37,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:38,005 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:54:39,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:40,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:54:40,770 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:54:41,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:43,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:45,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:47,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:49,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:51,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:53,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:55,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:56,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:54:56,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:54:57,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:54:59,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:01,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:03,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:05,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:07,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:08,084 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:55:09,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:11,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:11,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:55:11,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:55:13,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:15,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:17,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:19,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:21,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:23,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:25,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:26,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:55:26,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:55:27,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:29,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:31,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:33,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:35,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:37,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:38,160 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:55:39,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:41,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:42,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:55:42,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:55:43,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:45,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:47,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:49,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:51,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:53,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:55,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:57,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:55:57,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:55:57,590 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:55:59,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:01,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:03,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:05,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:07,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:08,237 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:56:09,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:11,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:13,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:13,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:56:13,326 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:56:15,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:18,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:20,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:22,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:24,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:26,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:28,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:28,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:56:28,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:56:30,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:32,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:34,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:36,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:38,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:38,310 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:56:40,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:42,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:43,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:56:43,796 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:56:44,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:46,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:48,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:50,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:52,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:54,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:56,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:58,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:56:59,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:56:59,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:57:00,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:02,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:04,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:06,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:08,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:08,376 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:57:10,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:12,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:14,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:14,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:57:14,446 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:57:16,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:18,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:20,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:22,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:24,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:26,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:28,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:57:29,678 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:57:30,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:32,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:35,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:37,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:38,443 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:57:39,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:41,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:43,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:45,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:57:45,075 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:57:45,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:47,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:49,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:51,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:53,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:55,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:57,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:57:59,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:00,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:58:00,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:58:01,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:03,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:05,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:07,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:08,516 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:58:09,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:11,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:13,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:15,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:15,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:58:15,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:58:17,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:19,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:21,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:23,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:25,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:27,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:29,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:31,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:58:31,053 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:58:31,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:34,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:36,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:38,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:38,588 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:58:40,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:42,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:44,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:46,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:46,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:58:46,280 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:58:48,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:50,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:52,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:54,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:56,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:58:58,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:00,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:01,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:59:01,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:59:02,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:04,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:06,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:08,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:08,660 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:59:10,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:12,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:14,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:16,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:16,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:59:16,914 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:59:18,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:20,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:22,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:24,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:26,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:28,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:30,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:59:32,170 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:59:32,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:34,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:36,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:38,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:38,730 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-23 23:59:40,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:42,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:44,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:46,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:47,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-23 23:59:47,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-23 23:59:48,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:50,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:52,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:54,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:56,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-23 23:59:59,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:01,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:02,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:00:02,760 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:00:03,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:05,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:07,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:08,812 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:00:09,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:11,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:13,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:15,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:17,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:18,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:00:18,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:00:19,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:21,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:23,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:25,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:27,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:29,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:31,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:33,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:33,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:00:33,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:00:35,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:37,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:38,883 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:00:39,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:41,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:43,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:45,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:47,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:48,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:00:48,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:00:49,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:51,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:53,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:55,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:57,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:00:59,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:01,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:03,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:04,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:01:04,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:01:05,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:07,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:08,959 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:01:09,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:11,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:13,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:15,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:17,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:19,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:19,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:01:19,762 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:01:21,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:23,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:25,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:27,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:30,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:32,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:34,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:35,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:01:35,051 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:01:36,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:38,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:39,032 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:01:40,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:42,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:44,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:46,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:48,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:50,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:50,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:01:50,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:01:52,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:54,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:56,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:01:58,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:00,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:02,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:04,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:05,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:02:05,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:02:06,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:08,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:09,104 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:02:10,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:12,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:14,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:16,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:18,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:20,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:21,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:02:21,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:02:22,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:24,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:26,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:28,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:30,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:32,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:34,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:36,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:36,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:02:36,301 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:02:38,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:39,180 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:02:40,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:43,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:45,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:47,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:49,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:51,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:51,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:02:51,754 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:02:53,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:55,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:57,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:02:59,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:01,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:03,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:05,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:07,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:07,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:03:07,509 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:03:09,245 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:03:09,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:11,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:13,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:15,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:17,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:19,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:21,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:22,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:03:22,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:03:23,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:25,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:27,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:29,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:31,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:33,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:35,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:37,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:38,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:03:38,145 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:03:39,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:39,308 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:03:41,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:43,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:45,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:47,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:49,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:51,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:53,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:53,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:03:53,498 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:03:55,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:57,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:03:59,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:01,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:03,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:05,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:07,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:08,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:04:08,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:04:09,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:09,386 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:04:11,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:13,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:15,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:17,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:19,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:21,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:23,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:24,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:04:24,130 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:04:25,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:27,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:29,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:31,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:33,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:35,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:37,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:39,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:04:39,460 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:04:39,698 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:04:40,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:42,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:44,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:46,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:48,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:50,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:52,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:54,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:54,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:04:54,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:04:56,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:04:58,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:00,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:02,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:04,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:06,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:08,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:09,540 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:05:09,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:05:09,937 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:05:10,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:12,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:14,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:16,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:18,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:20,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:22,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:24,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:25,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:05:25,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:05:26,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:28,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:30,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:32,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:34,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:36,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:38,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:39,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:05:40,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:40,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:05:40,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:05:42,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:44,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:46,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:49,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:51,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:53,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:55,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:55,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:05:55,700 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:05:57,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:05:59,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:01,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:03,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:05,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:07,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:09,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:09,691 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:06:10,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:06:10,922 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:06:11,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:13,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:15,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:17,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:19,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:21,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:23,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:25,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:26,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:06:26,316 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:06:27,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:29,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:31,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:33,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:35,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:37,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:39,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:39,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:06:41,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:41,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:06:41,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:06:43,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:45,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:47,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:49,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:51,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:53,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:55,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:06:56,894 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:06:57,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:06:59,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:01,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:03,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:05,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:07,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:09,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:09,834 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:07:11,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:12,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:07:12,131 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:07:13,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:15,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:17,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:19,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:21,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:23,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:26,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:27,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:07:27,410 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:07:28,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:30,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:32,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:34,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:36,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:38,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:39,910 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:07:40,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:42,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:42,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:07:42,680 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:07:44,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:46,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:48,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:50,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:52,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:54,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:56,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:07:57,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:07:57,917 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:07:58,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:00,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:02,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:04,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:06,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:08,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:09,982 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:08:10,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:12,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:13,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:08:13,219 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:08:14,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:16,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:18,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:20,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:22,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:24,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:26,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:28,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:28,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:08:28,494 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:08:30,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:32,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:34,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:36,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:38,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:40,045 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:08:40,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:42,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:43,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:08:43,763 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:08:44,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:46,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:48,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:50,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:52,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:54,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:56,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:58,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:08:59,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:08:59,084 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:09:01,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:03,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:05,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:07,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:09,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:10,111 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:09:11,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:13,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:14,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:09:14,310 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:09:15,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:17,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:19,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:21,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:23,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:25,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:27,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:29,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:29,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:09:29,583 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:09:31,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:33,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:35,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:37,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:39,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:40,186 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:09:41,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:43,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:44,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:09:44,911 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:09:45,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:47,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:49,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:51,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:53,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:55,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:57,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:09:59,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:00,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:10:00,246 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:10:01,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:03,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:05,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:08,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:10,260 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:10:10,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:12,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:14,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:15,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:10:15,482 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:10:16,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:18,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:20,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:22,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:24,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:26,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:28,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:30,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:30,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:10:30,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:10:32,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:34,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:36,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:38,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:40,335 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:10:40,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:42,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:44,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:45,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:10:45,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:10:46,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:48,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:50,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:52,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:54,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:56,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:10:58,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:00,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:01,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:11:01,332 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:11:02,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:04,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:06,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:08,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:10,409 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:11:10,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:12,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:14,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:16,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:16,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:11:16,734 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:11:18,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:20,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:22,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:24,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:26,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:29,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:31,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:31,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:11:31,971 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:11:33,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:35,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:37,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:39,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:40,476 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:11:41,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:43,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:45,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:47,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:11:47,229 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:11:47,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:49,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:51,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:53,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:55,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:57,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:11:59,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:01,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:02,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:12:02,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:12:03,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:05,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:07,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:09,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:10,552 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:12:11,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:13,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:15,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:17,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:17,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:12:17,697 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:12:19,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:21,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:23,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:25,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:27,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:29,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:31,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:33,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:12:33,034 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:12:33,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:35,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:37,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:39,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:40,628 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:12:41,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:43,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:45,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:48,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:12:48,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:12:48,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:50,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:52,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:54,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:56,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:12:58,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:00,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:02,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:03,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:13:03,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:13:04,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:06,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:08,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:10,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:10,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:13:12,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:14,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:16,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:18,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:18,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:13:18,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:13:20,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:22,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:24,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:26,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:28,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:30,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:32,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:34,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:13:34,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:13:34,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:36,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:38,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:40,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:40,763 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:13:42,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:44,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:46,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:48,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:49,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:13:49,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:13:50,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:52,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:54,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:56,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:13:58,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:14:00,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:14:02,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:14:04,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:14:04,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:14:04,821 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:14:06,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:14:08,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:14:10,842 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:14:20,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:14:20,153 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:14:35,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:14:35,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:14:40,916 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:14:50,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:14:50,723 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:15:05,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:15:05,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:15:07,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:15:07,877 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 00:15:07,928 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 00:15:08,002 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 00:15:08,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 00:15:09,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:15:10,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:15:11,011 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:15:15,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:15:21,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:15:21,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:15:35,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:15:36,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:15:36,485 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:15:41,093 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:15:51,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:15:51,745 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:16:06,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:16:06,989 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:16:11,177 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:16:22,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:16:22,230 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:16:37,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:16:37,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:16:41,259 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:16:52,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:16:52,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:17:08,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:17:08,062 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:17:11,344 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:17:23,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:17:23,290 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:17:38,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:17:38,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:17:41,429 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:17:53,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:17:53,757 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:18:08,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:18:08,990 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:18:11,514 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:18:24,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:18:24,244 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:18:39,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:18:39,475 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:18:41,597 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:18:54,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:18:54,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:19:09,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:19:09,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:19:11,680 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:19:25,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:19:25,214 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:19:39,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:40,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:40,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:19:40,478 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:19:41,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:41,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:41,765 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:19:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:43,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:44,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:44,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:45,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:46,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:46,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:47,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:49,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:49,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:50,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:50,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:51,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:52,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:52,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:53,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:55,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:19:55,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:19:55,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:56,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:56,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:57,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:19:58,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:58,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:19:59,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:00,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:00,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:01,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:02,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:02,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:03,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:05,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:06,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:06,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:09,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:11,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:11,833 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:20:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:13,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:14,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:15,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:20:15,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:20:22,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:28,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:31,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:20:31,021 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:20:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:31,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:32,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:36,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:39,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:40,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:41,899 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:20:42,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:46,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:20:46,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:20:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:46,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:48,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:48,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:49,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:50,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:52,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:53,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:20:56,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:20:58,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:00,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:00,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:21:01,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:21:02,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:02,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:04,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:08,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:09,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:10,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:11,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:11,968 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:21:12,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:14,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:15,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:15,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:16,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:21:16,799 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:21:18,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:20,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:22,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:23,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:23,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:24,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:25,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:25,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:26,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:28,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:30,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:32,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:21:32,038 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:21:32,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:33,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:33,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:33,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:33,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:34,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:35,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:36,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:38,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:39,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:39,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:40,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:41,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:41,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:42,036 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:21:42,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:43,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:43,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:44,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:45,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:45,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:46,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:47,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:21:47,365 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:21:47,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:47,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:48,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:49,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:49,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:50,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:52,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:53,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:54,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:55,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:55,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:56,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:58,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:21:58,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:21:58,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:00,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:01,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:01,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:02,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:22:02,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:22:02,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:04,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:04,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:04,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:06,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:08,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:12,109 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:22:12,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:12,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:17,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:22:17,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:22:18,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:21,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:21,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:24,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:28,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:32,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:33,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:22:33,356 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:22:36,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:38,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:38,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:38,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:38,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:38,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:39,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:40,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:42,183 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:22:42,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:42,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:45,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:47,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:47,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:48,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:22:48,588 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:22:51,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:51,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:53,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:53,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:54,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:55,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:22:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:22:57,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:01,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:01,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:03,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:03,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:23:03,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:23:05,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:08,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:08,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:09,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:11,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:11,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:12,255 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:23:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:12,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:13,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:14,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:14,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:17,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:18,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:19,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:23:19,166 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:23:19,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:21,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:23,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:25,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:27,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:28,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:29,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:31,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:33,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:33,826 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 00:23:33,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:34,042 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 00:23:34,121 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 00:23:34,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:23:34,456 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:23:34,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 00:23:35,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:37,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:39,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:39,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:40,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:41,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:42,337 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:23:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:42,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:43,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:45,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:45,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:47,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:47,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:47,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:49,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:23:49,752 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:23:49,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:50,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:50,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:51,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:53,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:53,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:54,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:54,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:55,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:56,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:56,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:57,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:23:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:58,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:23:59,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:00,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:00,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:01,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:02,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:02,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:03,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:05,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:24:05,023 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:24:05,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:07,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:12,402 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:24:15,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:19,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:19,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:20,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:24:20,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:24:21,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:24,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:24,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:25,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:27,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:27,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:28,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:29,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:33,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:33,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:35,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:24:35,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:24:37,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:39,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:42,467 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:24:43,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:45,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:47,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:49,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:50,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:51,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:24:51,040 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:24:52,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:53,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:53,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:54,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:56,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:56,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:58,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:24:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:24:59,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:00,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:02,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:04,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:25:06,304 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:25:06,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:08,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:09,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:09,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:10,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:12,543 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:25:12,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:14,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:15,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:15,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:16,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:18,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:19,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:19,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:20,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:21,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:25:21,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:25:22,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:23,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:23,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:26,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:27,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:27,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:28,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:29,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:29,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:30,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:32,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:33,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:33,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:34,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:35,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:25:36,927 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:25:36,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:38,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:38,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:39,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:39,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:40,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:42,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:42,613 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:25:42,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:44,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:44,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:45,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:46,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:47,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:48,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:49,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:50,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:51,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:25:52,228 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:25:52,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:52,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:53,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:53,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:54,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:55,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:55,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:56,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:57,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:57,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:25:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:25:58,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:00,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:02,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:07,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:26:07,504 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:26:08,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:12,682 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:26:14,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:18,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:18,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:22,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:26:22,836 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:26:22,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:24,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:25,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:27,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:28,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:32,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:32,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:36,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:38,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:26:38,209 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:26:39,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:40,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:40,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:42,747 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:26:43,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:44,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:45,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:48,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:48,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:49,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:52,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:52,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:53,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:26:53,459 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:26:54,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:26:55,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:55,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:26:58,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:00,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:02,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:04,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:05,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:05,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:06,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:08,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:27:08,786 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:27:10,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:12,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:12,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:12,816 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:27:13,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:13,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:14,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:14,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:16,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:18,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:19,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:19,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:20,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,196 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 00:27:20,399 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 00:27:20,474 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 00:27:20,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:20,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:21,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 00:27:22,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:23,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:24,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:24,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:27:24,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:27:25,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:25,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:26,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:28,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:29,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:29,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:30,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:32,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:33,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:33,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:34,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:36,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:37,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:38,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:27:39,313 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:27:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:39,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:40,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:41,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:42,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:42,941 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:27:44,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:44,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:45,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:46,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:48,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:49,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:50,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:51,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:27:52,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:54,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:27:54,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:27:54,564 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:28:04,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:08,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:09,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:28:09,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:28:12,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:13,010 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:28:15,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:15,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:16,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:20,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:24,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:25,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:28:25,058 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:28:26,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:30,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:30,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:30,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:34,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:34,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:36,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:38,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:40,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:28:40,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:28:42,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:42,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:42,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:43,077 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:28:44,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:46,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:46,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:48,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:49,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:49,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:50,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:50,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:52,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:53,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:53,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:54,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:28:55,779 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:28:55,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:55,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:28:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:56,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:28:58,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:00,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:01,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:01,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:03,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:05,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:05,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:05,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:07,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:07,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:09,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:10,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:10,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:29:11,017 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:29:11,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:11,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:13,145 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:29:13,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:13,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:14,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:15,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:17,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:18,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:18,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:19,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:20,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:20,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:21,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:23,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:25,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:26,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:29:26,294 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:29:27,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:28,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:28,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:29,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:31,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:31,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:31,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:32,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:33,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:33,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:35,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:35,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:36,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:37,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:38,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:38,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:39,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:39,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:40,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:41,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:29:41,571 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:29:41,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:43,218 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:29:43,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:43,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:44,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:45,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:45,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:47,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:49,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:29:50,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:50,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:51,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:53,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:29:56,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:29:56,812 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:29:59,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:02,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:05,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:09,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:12,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:30:12,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:30:13,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:13,293 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:30:17,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:20,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:20,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:21,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:25,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:27,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:27,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:30:27,467 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:30:31,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:31,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:31,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:33,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:37,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:38,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:39,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:42,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:30:42,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:30:43,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:43,357 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:30:45,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:49,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:50,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:50,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:51,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:51,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:53,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:55,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:30:55,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:55,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:56,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:57,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:30:58,194 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:30:58,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:58,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:30:59,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:00,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:00,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:02,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:04,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:04,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:06,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:08,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:10,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:10,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:10,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:11,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:11,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:11,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:11,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:12,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:12,599 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 00:31:12,828 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 00:31:12,909 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 00:31:13,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 00:31:13,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:13,474 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:31:13,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:31:13,659 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:31:14,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:14,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:16,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:18,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:20,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:20,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:20,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:22,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:22,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:23,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:24,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:26,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:27,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:27,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:28,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:31:28,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:31:30,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:30,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:31,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:32,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:32,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:34,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:35,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:35,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:36,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:36,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:37,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:38,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:39,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:39,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:40,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:41,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:42,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,545 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:31:43,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:43,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:44,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:31:44,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:31:44,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:44,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:46,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:48,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:56,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:31:57,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:57,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:31:59,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:31:59,452 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:32:01,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:01,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:02,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:03,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:03,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:05,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:06,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:10,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:13,615 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:32:14,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:14,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:32:14,823 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:32:16,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:16,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:18,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:22,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:24,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:26,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:28,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:30,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:32:30,088 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:32:32,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:34,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:36,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:36,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:36,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:40,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:42,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:43,687 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:32:44,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:44,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:44,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:32:45,317 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:32:48,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:48,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:48,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:50,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:52,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:54,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:56,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:32:58,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:58,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:32:59,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:00,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:00,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:33:00,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:33:02,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:04,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:04,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:05,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:06,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:08,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:08,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:08,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:09,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:10,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:11,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:12,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:12,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:13,757 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:33:14,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:15,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:33:15,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:33:16,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:17,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:17,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:18,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:20,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:20,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:20,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:21,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:23,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:25,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:27,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:28,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:29,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:30,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:30,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:33:31,134 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:33:31,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:31,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:31,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:32,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:33,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:33,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:34,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:35,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:35,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:36,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:37,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:37,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:38,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:39,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:40,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:40,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:41,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:41,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:43,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:43,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:33:46,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:33:46,607 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:33:50,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:50,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:53,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:54,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:33:57,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:57,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:33:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:01,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:01,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:34:01,896 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:34:05,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:06,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:06,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:07,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:08,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:09,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:13,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:13,891 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:34:17,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:34:17,126 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:34:17,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:19,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:21,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:23,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:24,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:24,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:27,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:29,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:31,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:32,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:34:32,350 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:34:34,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:34,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:35,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:37,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:41,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:42,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:42,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:43,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:43,961 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:34:45,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:47,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:47,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:47,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:34:47,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:34:48,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:48,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:49,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:51,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:53,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:55,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:56,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:56,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:57,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:34:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:58,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:34:59,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:00,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:00,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:01,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:02,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:35:02,802 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:35:03,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:04,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:04,989 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 00:35:05,199 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 00:35:05,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:05,304 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 00:35:05,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:05,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 00:35:06,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:06,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:06,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:07,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:08,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:08,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:09,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:12,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:12,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:12,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,090 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:35:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:14,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:15,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:16,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:17,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:17,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:18,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:35:18,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:35:18,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:19,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:19,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:20,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:22,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:23,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:23,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:24,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:25,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:25,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:26,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:28,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:29,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:29,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:30,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:31,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:31,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:32,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:32,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:33,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:35:33,396 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:35:34,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:35,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:35,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:36,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:37,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:38,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:38,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:40,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:44,159 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:35:47,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:47,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:48,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:35:48,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:35:49,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:49,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:50,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:52,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:52,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:54,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:56,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:35:59,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:35:59,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:00,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:01,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:01,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:04,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:36:04,161 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:36:04,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:05,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:05,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:08,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:08,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:10,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:12,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:12,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:14,232 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:36:14,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:18,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:19,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:19,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:36:19,429 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:36:20,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:22,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:22,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:23,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:24,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:26,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:30,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:33,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:34,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:36:34,712 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:36:36,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:37,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:37,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:39,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:41,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:44,307 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:36:45,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:46,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:46,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:47,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:49,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:49,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:36:49,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:36:51,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:53,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:54,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:54,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,074 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:55,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:59,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:36:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:36:59,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:01,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:02,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:02,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:03,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:03,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:05,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:37:05,186 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:37:05,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:06,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:07,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:08,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:08,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:09,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:09,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:11,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:13,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:13,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:14,380 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:37:14,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:14,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:15,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:17,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:17,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:17,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:18,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:19,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:20,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:37:20,535 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:37:21,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:22,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:23,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:25,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:26,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:26,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:27,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:29,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:30,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:30,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:31,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:32,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:33,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:33,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:34,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:35,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:35,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:37:35,753 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:37:36,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:36,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:37,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:39,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:44,445 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:37:49,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:49,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:51,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:37:51,011 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:37:53,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:37:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:57,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:58,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:37:59,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:02,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:02,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:03,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:05,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,801 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:05,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:06,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:38:06,304 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:38:07,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:09,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:09,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:09,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:13,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:13,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:14,509 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:38:16,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:16,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:17,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:19,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:21,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:21,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:38:21,665 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:38:23,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:24,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:25,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:29,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:32,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:36,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:37,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:38:37,114 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:38:38,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:40,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:41,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:41,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:42,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:42,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:44,579 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:38:46,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:48,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:48,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:48,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:50,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:52,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:52,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:38:52,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:38:54,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:56,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:38:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:57,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:38:58,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:00,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:00,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:00,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:00,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:00,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 00:39:01,418 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 00:39:01,534 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 00:39:01,591 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 00:39:02,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 00:39:02,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:03,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:04,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:06,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:07,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:39:07,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:39:08,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:10,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:12,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:14,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:14,665 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:39:16,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:18,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:20,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:22,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:22,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:39:22,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:39:24,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:26,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:28,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:30,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:32,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:34,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:36,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:38,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:39:38,207 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:39:38,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:40,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:42,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:44,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:44,739 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:39:47,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:49,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:51,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:53,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:53,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:39:53,424 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:39:55,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:57,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:39:59,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:01,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:03,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:05,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:07,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:08,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:40:08,743 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:40:09,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:11,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:13,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:14,807 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:40:15,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:17,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:19,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:21,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:23,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:23,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:40:23,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:40:25,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:27,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:29,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:31,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:33,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:35,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:37,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:40:39,207 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:40:39,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:41,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:43,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:44,877 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:40:45,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:47,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:49,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:51,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:53,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:54,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:40:54,503 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:40:55,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:57,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:40:59,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:01,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:03,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:05,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:07,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:09,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:09,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:41:09,744 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:41:11,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:13,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:14,951 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:41:15,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:17,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:20,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:22,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:24,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:41:25,013 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:41:26,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:28,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:30,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:32,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:34,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:36,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:38,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:40,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:41:40,294 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:41:40,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:42,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:44,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:45,023 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:41:46,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:48,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:50,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:52,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:54,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:55,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:41:55,526 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:41:56,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:41:58,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:00,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:02,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:04,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:06,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:08,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:10,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:42:10,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:42:12,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:14,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:15,097 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:42:16,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:18,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:20,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:22,398 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:24,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:26,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:42:26,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:42:26,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:28,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:30,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:32,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:34,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:36,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:38,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:40,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:41,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:42:41,267 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:42:42,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:44,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:45,172 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:42:46,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:48,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:50,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:53,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:55,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:56,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:42:56,584 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:42:57,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:42:59,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:01,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:03,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:05,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:07,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:09,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:11,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:11,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:43:11,846 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:43:13,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:15,248 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:43:15,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:17,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:19,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:21,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:23,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:25,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:27,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:43:27,068 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:43:27,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:29,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:31,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:33,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:35,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:37,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:39,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:41,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:42,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:43:42,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:43:43,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:45,325 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:43:45,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:47,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:49,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:51,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:53,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:55,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:57,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:43:57,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:43:57,741 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:43:59,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:01,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:03,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:05,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:08,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:10,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:12,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:13,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:44:13,010 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:44:14,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:15,398 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:44:16,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:18,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:20,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:22,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:24,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:26,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:28,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:44:28,234 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:44:28,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:30,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:32,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:34,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:36,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:38,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:40,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:42,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:43,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:44:43,615 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:44:44,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:45,471 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:44:46,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:48,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:50,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:52,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:54,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:56,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:58,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:44:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:44:58,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:45:00,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:02,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:04,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:06,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:09,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:11,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:13,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:14,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:45:14,184 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:45:15,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:15,535 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:45:17,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:19,471 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:21,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:23,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:25,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:27,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:29,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:45:29,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:45:29,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:31,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:33,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:35,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:37,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:39,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:41,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:43,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:44,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:45:44,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:45:45,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:45,612 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:45:47,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:49,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:51,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:53,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:55,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:57,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:59,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:45:59,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:45:59,972 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:46:01,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:03,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:05,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:07,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:09,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:11,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:13,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:15,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:46:15,263 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:46:15,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:15,688 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:46:17,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:19,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:21,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:23,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:25,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:27,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:29,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:30,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:46:30,595 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:46:31,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:34,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:36,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:38,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:40,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:42,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:44,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:45,762 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:46:45,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:46:45,967 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:46:46,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:48,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:50,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:52,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:54,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:56,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:46:58,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:00,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:01,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:47:01,199 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:47:02,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:04,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:06,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:08,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:10,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:12,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:14,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:15,839 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:47:16,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:47:16,471 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:47:16,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:18,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:20,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:22,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:24,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:26,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:28,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:30,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:31,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:47:31,739 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:47:32,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:34,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:36,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:38,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:40,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:42,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:44,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:45,911 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:47:46,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:46,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:47:46,965 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:47:48,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:50,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:52,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:54,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:56,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:47:59,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:01,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:02,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:48:02,193 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:48:03,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:05,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:07,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:09,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:11,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:13,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:15,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:15,986 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:48:17,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:17,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:48:17,619 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:48:19,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:21,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:23,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:25,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:27,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:29,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:31,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:32,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:48:32,942 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:48:33,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:35,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:37,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:39,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:41,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:43,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:45,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:46,060 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:48:47,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:48,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:48:48,266 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:48:49,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:51,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:53,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:55,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:57,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:48:59,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:01,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:03,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:49:03,487 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:49:03,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:05,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:07,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:09,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:12,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:14,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:16,138 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:49:16,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:18,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:18,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:49:18,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:49:20,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:22,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:24,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:26,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:28,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:30,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:32,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:33,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:49:33,952 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:49:34,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:36,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:38,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:40,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:42,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:44,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:46,210 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:49:46,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:48,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:49,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:49:49,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:49:50,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:52,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:54,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:56,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:49:58,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:00,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:02,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:04,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:50:04,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:50:04,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:06,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:08,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:10,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:12,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:14,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:16,287 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:50:16,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:18,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:19,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:50:19,705 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:50:20,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:22,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:24,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:26,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:28,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:30,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:32,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:34,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:34,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:50:34,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:50:36,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:38,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:41,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:43,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:45,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:46,363 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:50:47,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:49,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:50,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:50:50,169 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:50:51,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:53,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:55,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:57,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:50:59,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:01,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:03,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:05,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:51:05,449 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:51:05,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:07,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:09,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:11,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:13,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:15,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:16,441 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:51:17,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:19,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:20,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:51:20,704 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:51:21,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:23,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:25,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:27,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:29,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:31,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:33,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:35,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:35,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:51:35,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:51:37,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:39,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:41,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:43,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:45,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:46,516 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:51:47,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:50,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:51,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:51:51,208 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:51:52,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:54,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:56,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:51:58,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:00,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:02,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:04,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:06,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:52:06,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:52:06,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:08,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:10,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:12,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:14,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:16,590 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:52:16,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:18,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:20,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:21,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:52:21,774 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:52:22,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:24,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:26,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:28,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:30,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:32,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:34,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:36,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:37,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:52:37,044 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:52:38,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:40,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:42,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:44,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:46,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:46,667 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:52:48,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:50,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:52,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:52:52,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:52:52,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:54,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:56,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:52:58,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:00,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:02,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:05,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:07,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:53:07,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:53:07,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:09,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:11,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:13,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:15,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:16,745 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:53:17,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:19,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:21,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:22,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:53:22,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:53:23,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:25,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:27,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:29,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:31,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:33,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:35,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:37,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:38,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:53:38,260 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:53:39,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:41,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:43,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:45,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:46,821 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:53:47,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:49,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:51,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:53,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:53:53,496 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:53:53,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:55,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:57,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:53:59,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:01,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:03,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:05,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:07,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:08,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:54:08,718 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:54:09,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:11,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:13,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:15,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:16,897 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:54:17,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:19,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:21,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:23,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:23,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:54:23,947 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:54:25,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:27,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:29,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:31,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:33,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:35,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:37,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:39,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:54:39,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:54:39,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:41,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:44,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:46,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:46,974 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:54:48,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:50,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:52,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:54,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:54:54,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:54:54,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:56,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:54:58,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:00,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:02,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:04,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:06,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:08,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:09,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:55:09,908 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:55:10,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:12,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:14,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:16,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:17,052 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:55:18,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:20,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:22,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:24,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:25,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:55:25,146 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:55:26,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:28,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:30,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:32,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:34,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:36,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:38,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:40,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:55:40,470 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:55:40,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:42,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:44,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:46,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:47,129 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:55:48,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:50,738 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:52,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:54,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:55,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:55:55,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:55:56,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:55:58,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:00,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:02,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:04,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:06,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:08,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:10,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:11,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:56:11,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:56:12,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:14,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:16,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:17,203 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:56:18,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:20,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:22,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:24,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:26,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:56:26,286 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:56:26,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:28,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:30,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:32,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:34,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:36,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:38,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:40,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:41,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:56:41,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:56:42,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:44,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:46,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:47,279 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:56:48,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:50,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:52,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:54,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:56,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:56:56,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:56:56,879 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:56:58,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:00,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:02,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:04,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:07,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:09,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:11,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:12,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:57:12,252 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:57:13,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:15,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:17,356 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:57:17,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:19,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:21,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:23,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:25,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:27,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:57:27,481 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:57:27,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:29,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:31,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:33,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:35,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:37,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:39,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:41,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:42,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:57:42,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:57:43,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:45,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:47,434 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:57:47,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:49,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:51,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:53,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:55,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:57,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:57:57,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:57:57,969 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:57:59,791 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:01,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:03,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:05,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:08,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:10,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:12,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:58:13,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:58:14,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:16,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:17,510 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:58:18,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:20,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:22,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:24,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:26,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:28,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:58:28,652 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:58:28,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:30,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:32,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:34,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:36,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:38,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:40,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:42,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:43,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:58:43,935 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:58:44,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:46,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:47,585 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:58:48,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:50,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:52,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:54,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:56,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:58,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:58:59,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:58:59,165 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:59:00,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:02,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:04,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:06,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:08,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:10,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:12,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:14,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:59:14,447 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:59:14,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:16,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:17,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:59:18,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:20,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:22,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:24,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:26,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:28,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:29,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:59:29,796 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:59:30,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:32,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:34,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:36,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:38,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:40,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:42,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:44,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 00:59:45,082 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 00:59:46,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:47,732 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 00:59:48,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:50,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:52,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:54,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:56,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 00:59:58,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:00:00,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:00:00,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:02,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:04,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:06,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:08,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:10,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:13,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:15,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:00:15,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:00:15,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:17,806 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:00:17,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:19,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:21,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:23,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:25,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:27,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:29,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:00:30,953 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:00:31,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:33,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:35,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:37,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:39,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:41,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:43,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:45,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:46,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:00:46,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:00:47,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:47,883 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:00:49,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:51,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:53,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:55,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:57,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:00:59,864 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:01,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:01:01,472 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:01:01,865 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:03,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:05,866 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:07,867 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:09,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:11,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:13,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:15,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:16,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:01:16,696 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:01:17,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:17,957 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:01:19,871 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:21,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:23,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:25,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:28,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:30,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:31,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:01:31,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:01:32,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:34,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:36,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:38,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:40,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:42,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:44,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:46,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:01:47,226 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:01:48,034 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:01:48,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:50,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:52,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:54,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:56,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:01:58,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:00,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:02,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:02:02,454 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:02:02,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:04,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:06,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:08,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:10,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:12,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:14,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:16,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:17,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:02:17,703 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:02:18,111 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:02:18,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:20,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:22,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:24,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:26,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:28,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:30,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:32,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:32,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:02:32,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:02:34,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:36,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:39,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:41,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:43,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:45,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:47,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:48,188 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:02:48,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:02:48,300 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:02:49,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:51,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:53,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:55,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:57,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:02:59,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:01,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:03,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:03:03,533 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:03:03,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:05,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:07,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:09,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:11,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:13,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:15,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:17,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:18,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:03:18,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:03:18,895 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:03:19,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:21,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:23,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:25,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:27,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:29,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:31,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:33,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:34,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:03:34,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:03:35,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:37,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:39,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:41,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:43,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:45,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:47,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:48,338 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:03:49,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:03:49,379 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:03:49,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:52,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:54,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:56,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:03:58,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:00,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:02,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:04,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:04:04,603 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:04:04,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:06,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:08,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:10,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:12,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:14,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:16,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:18,416 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:04:18,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:19,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:04:19,996 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:04:20,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:22,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:24,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:26,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:28,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:30,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:32,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:34,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:35,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:04:35,382 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:04:36,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:38,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:40,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:42,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:44,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:46,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:48,487 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:04:48,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:50,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:04:50,621 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:04:50,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:52,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:54,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:56,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:04:58,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:00,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:02,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:04,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:05,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:05:05,946 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:05:06,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:08,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:10,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:12,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:14,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:17,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:18,559 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:05:19,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:21,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:05:21,188 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:05:21,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:23,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:25,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:27,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:29,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:31,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:33,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:35,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:36,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:05:36,486 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:05:37,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:39,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:41,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:43,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:45,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:47,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:48,636 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:05:49,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:51,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:05:51,749 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:05:51,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:53,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:55,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:57,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:05:59,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:01,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:03,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:05,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:07,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:06:07,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:06:07,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:09,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:11,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:13,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:15,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:17,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:18,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:06:19,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:21,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:22,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:06:22,352 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:06:23,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:25,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:27,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:29,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:31,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:33,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:36,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:37,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:06:37,582 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:06:38,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:40,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:42,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:44,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:46,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:48,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:48,791 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:06:50,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:52,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:52,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:06:52,797 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:06:55,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:57,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:06:59,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:01,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:03,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:05,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:07,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:08,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:07:08,101 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:07:09,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:11,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:13,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:15,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:17,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:18,867 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:07:19,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:21,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:23,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:23,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:07:23,325 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:07:25,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:27,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:29,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:31,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:33,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:35,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:37,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:38,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:07:38,582 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:07:39,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:41,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:43,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:45,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:47,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:48,943 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:07:49,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:51,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:53,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:53,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:07:53,931 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:07:55,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:57,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:07:59,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:01,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:03,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:05,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:07,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:09,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:09,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:08:09,258 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:08:11,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:13,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:15,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:17,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:19,019 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:08:20,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:22,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:24,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:24,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:08:24,558 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:08:26,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:28,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:30,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:32,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:34,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:36,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:38,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:39,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:08:39,817 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:08:40,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:42,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:44,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:46,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:48,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:49,094 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:08:50,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:52,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:54,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:55,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:08:55,045 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:08:56,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:08:58,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:00,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:02,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:04,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:06,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:08,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:10,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:10,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:09:10,274 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:09:12,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:14,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:16,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:18,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:19,168 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:09:20,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:22,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:25,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:25,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:09:25,576 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:09:27,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:29,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:31,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:33,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:35,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:37,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:39,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:40,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:09:40,803 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:09:41,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:43,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:45,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:47,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:49,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:49,244 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:09:51,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:53,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:55,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:56,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:09:56,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:09:57,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:09:59,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:01,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:03,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:05,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:07,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:09,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:11,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:11,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:10:11,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:10:13,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:15,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:17,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:19,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:19,320 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:10:21,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:23,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:25,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:26,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:10:26,658 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:10:27,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:29,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:31,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:33,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:35,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:37,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:39,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:41,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:41,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:10:41,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:10:43,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:45,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:47,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:49,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:49,397 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:10:51,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:53,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:55,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:57,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:10:57,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:10:57,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:10:59,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:01,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:03,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:05,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:08,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:10,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:12,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:12,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:11:12,463 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:11:14,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:16,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:18,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:19,473 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:11:20,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:22,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:24,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:26,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:27,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:11:27,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:11:28,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:30,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:32,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:34,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:36,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:38,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:40,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:42,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:43,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:11:43,033 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:11:44,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:46,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:48,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:49,550 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:11:50,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:52,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:54,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:56,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:58,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:11:58,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:11:58,307 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:12:00,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:02,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:04,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:06,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:08,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:10,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:12,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:13,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:12:13,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:12:14,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:16,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:18,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:19,625 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:12:20,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:22,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:24,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:26,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:28,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:28,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:12:28,837 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:12:30,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:32,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:34,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:36,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:38,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:40,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:42,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:44,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:44,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:12:44,164 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:12:46,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:48,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:49,701 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:12:50,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:53,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:55,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:57,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:59,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:12:59,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:12:59,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:13:01,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:03,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:05,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:07,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:09,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:11,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:13,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:14,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:13:14,650 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:13:15,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:17,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:19,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:19,775 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:13:21,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:23,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:25,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:27,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:29,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:29,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:13:29,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:13:31,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:33,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:35,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:37,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:39,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:41,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:43,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:45,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:45,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:13:45,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:13:47,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:49,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:49,849 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:13:51,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:53,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:55,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:57,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:13:59,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:00,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:14:00,521 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:14:01,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:03,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:05,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:07,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:09,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:11,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:14,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:15,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:14:15,822 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:14:16,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:18,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:19,926 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:14:20,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:22,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:24,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:26,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:28,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:30,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:31,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:14:31,096 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:14:32,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:34,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:36,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:38,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:40,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:42,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:44,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:46,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:46,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:14:46,402 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:14:48,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:50,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:14:50,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:52,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:54,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:56,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:14:58,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:00,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:01,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:15:01,630 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:15:02,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:04,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:06,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:08,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:10,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:12,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:14,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:16,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:16,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:15:16,928 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:15:18,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:20,075 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:15:20,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:22,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:24,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:26,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:28,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:30,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:32,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:32,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:15:32,259 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:15:34,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:36,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:38,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:41,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:43,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:45,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:47,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:47,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:15:47,572 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:15:49,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:50,149 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:15:51,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:53,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:55,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:57,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:15:59,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:01,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:02,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:16:02,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:16:03,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:05,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:07,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:09,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:11,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:13,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:15,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:17,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:18,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:16:18,113 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:16:19,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:20,225 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:16:21,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:23,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:25,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:27,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:29,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:31,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:33,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:33,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:16:33,514 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:16:35,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:37,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:39,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:41,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:43,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:45,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:47,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:48,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:16:48,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:16:49,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:50,296 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:16:51,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:53,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:55,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:57,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:16:59,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:01,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:03,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:04,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:17:04,141 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:17:05,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:08,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:10,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:12,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:14,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:16,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:18,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:19,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:17:19,495 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:17:20,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:20,366 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:17:22,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:24,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:26,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:28,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:30,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:32,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:34,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:34,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:17:34,811 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:17:36,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:38,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:40,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:42,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:44,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:46,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:48,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:17:50,070 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:17:50,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:50,423 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:17:52,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:54,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:56,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:17:58,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:00,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:02,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:04,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:05,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:18:05,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:18:06,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:08,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:10,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:12,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:14,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:16,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:18,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:20,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:20,495 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:18:20,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:18:20,541 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:18:22,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:24,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:26,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:28,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:30,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:32,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:34,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:35,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:18:35,818 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:18:36,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:38,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:40,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:42,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:44,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:46,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:48,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:50,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:50,570 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:18:51,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:18:51,047 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:18:52,295 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:54,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:56,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:18:58,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:00,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:02,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:04,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:06,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:19:06,333 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:19:07,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:09,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:11,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:13,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:15,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:17,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:19,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:20,644 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:19:21,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:21,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:19:21,577 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:19:23,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:25,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:27,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:29,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:31,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:33,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:35,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:36,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:19:36,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:19:37,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:39,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:41,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:43,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:45,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:47,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:49,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:50,710 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:19:51,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:52,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:19:52,309 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:19:53,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:55,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:57,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:19:59,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:01,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:03,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:05,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:07,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:07,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:20:07,604 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:20:09,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:11,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:13,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:16,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:18,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:20,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:20,772 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:20:22,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:22,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:20:22,832 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:20:24,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:26,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:28,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:30,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:32,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:34,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:36,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:38,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:20:38,167 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:20:38,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:40,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:42,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:44,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:46,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:48,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:50,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:50,847 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:20:52,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:53,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:20:53,399 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:20:54,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:56,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:20:58,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:00,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:02,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:04,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:06,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:08,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:08,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:21:08,623 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:21:10,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:21:20,906 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:21:23,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:21:23,850 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:21:39,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:21:39,097 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:21:50,966 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:21:54,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:21:54,391 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:22:07,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:22:07,258 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 01:22:07,311 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 01:22:07,388 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 01:22:08,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 01:22:08,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:22:09,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:22:09,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:22:09,644 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:22:14,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:22:21,050 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:22:24,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:22:24,904 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:22:32,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:22:40,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:22:40,132 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:22:51,131 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:22:55,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:22:55,366 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:23:10,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:23:10,600 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:23:21,213 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:23:25,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:23:25,833 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:23:41,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:23:41,064 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:23:51,300 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:23:56,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:23:56,291 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:24:11,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:24:11,520 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:24:21,385 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:24:26,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:24:26,746 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:24:41,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:24:41,975 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:24:51,468 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:24:57,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:24:57,235 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:25:12,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:25:12,493 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:25:21,552 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:25:27,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:25:27,755 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:25:42,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:25:42,984 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:25:51,635 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:25:58,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:25:58,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:26:13,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:26:13,451 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:26:21,718 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:26:28,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:26:28,684 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:26:41,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:42,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:42,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:43,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:26:43,974 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:26:45,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:46,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:46,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:47,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:48,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:48,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:49,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:50,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:50,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:51,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:51,800 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:26:52,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:52,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:53,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:55,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:57,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:57,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:57,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:26:59,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:26:59,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:26:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:26:59,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,129 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:01,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:02,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:03,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:05,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:06,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:06,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:07,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,295 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,300 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:09,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:09,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:10,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:11,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:12,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:13,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:14,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:27:14,579 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:27:15,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:17,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:17,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:21,872 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:27:22,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:22,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:25,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:25,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:26,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:29,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:27:29,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:27:30,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:30,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:31,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:32,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:32,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:35,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:39,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:40,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:40,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:43,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:45,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:27:45,105 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:27:45,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:45,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:47,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:49,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:51,937 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:27:52,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:52,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:53,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:54,296 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:54,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:55,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:27:56,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:58,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:27:59,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:00,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:28:00,347 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:28:01,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:02,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:02,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:04,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:05,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:07,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:07,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:09,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:13,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:15,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:15,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:28:15,682 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:28:17,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:21,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:21,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:22,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:28:23,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:25,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:27,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:29,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:30,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:28:30,948 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:28:31,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:33,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:35,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:36,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:36,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:37,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:38,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:40,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:40,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:40,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:42,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:42,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:43,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:44,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:45,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:45,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:46,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:28:46,238 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:28:46,569 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:48,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:48,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:49,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:49,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:50,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:51,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,073 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:28:52,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:52,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:53,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:53,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:54,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:56,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:57,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:57,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:58,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:28:59,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:28:59,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:00,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:01,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:01,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:29:01,468 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:29:02,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:03,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:03,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:04,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:06,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:08,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:08,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:08,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:09,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:10,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:12,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:14,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:14,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:16,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:29:16,698 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:29:20,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:22,146 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:29:24,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:26,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:26,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:27,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:28,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:29,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:31,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:29:32,046 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:29:32,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:32,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:32,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:34,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:36,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:37,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:37,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:40,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:42,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:42,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:44,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:45,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:45,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:46,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:47,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:29:47,323 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:29:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:48,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:50,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:52,220 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:29:52,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:53,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:53,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:54,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:55,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:56,598 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:29:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:57,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:29:58,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:02,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:30:02,673 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:30:04,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:07,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:08,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:10,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:10,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:10,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:10,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:10,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:10,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:10,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:11,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:12,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:15,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:16,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:16,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:17,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:17,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:30:17,943 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:30:21,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:22,293 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:30:23,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:24,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:24,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:25,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:27,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:29,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:29,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:29,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:30,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:31,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:31,205 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 01:30:31,418 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 01:30:31,496 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 01:30:31,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 01:30:31,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:32,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:33,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:30:33,247 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:30:33,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:33,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:33,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:35,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:37,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:38,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:38,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:39,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:40,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:40,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:41,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:43,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:44,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:45,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:45,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:47,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:47,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:30:48,552 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:30:48,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:48,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:49,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:50,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:50,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:51,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:52,377 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:30:53,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:54,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:54,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:55,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:56,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:56,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:30:57,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:57,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:58,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:30:59,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:01,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:02,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,731 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,732 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,734 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:02,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:03,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:31:03,824 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:31:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:03,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:05,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:13,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:18,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:19,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:31:19,076 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:31:19,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:21,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:21,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:22,447 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:31:23,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:26,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:26,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:27,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:28,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:28,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,737 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,743 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,753 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,754 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:30,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:31,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,294 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:32,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:31:34,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:31:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,665 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:34,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:35,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:37,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:41,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:42,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:42,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:45,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:45,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:47,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:48,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:48,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:31:49,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:31:50,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,752 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:50,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:51,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:52,513 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:31:53,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:53,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:53,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:54,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:31:57,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:31:59,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:01,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:03,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:04,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:04,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:32:04,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:32:07,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:09,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:09,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:11,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:11,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:14,659 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:16,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:16,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:18,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:20,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:32:20,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:32:20,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:22,585 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:32:22,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:24,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:25,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:25,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:26,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:27,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:28,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:30,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:32,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:32,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:34,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:35,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:35,435 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:32:35,435 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:32:36,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:37,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:37,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,616 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:38,669 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:40,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:42,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:42,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:43,898 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:44,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:46,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:46,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:46,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:47,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:48,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:50,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:50,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:50,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:32:50,716 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:32:51,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:51,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:52,659 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:32:52,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:53,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:53,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:54,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:55,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:55,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:32:56,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:32:58,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:00,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:05,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:33:05,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:33:06,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:06,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:08,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:09,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,742 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,750 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:09,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:13,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,414 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,423 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:14,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:18,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:21,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:33:21,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:33:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:22,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:22,735 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:33:23,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:23,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:26,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:27,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,203 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,565 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:27,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:28,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:29,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:29,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:30,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:32,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:33,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:33,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:36,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:33:36,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:33:36,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:37,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:38,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:39,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:39,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:42,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:43,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:43,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:44,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:45,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,757 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:45,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:48,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:50,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:51,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:51,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:33:51,672 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:33:52,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:52,809 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:33:52,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:52,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:55,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:56,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:33:57,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:57,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:33:58,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:01,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:02,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:02,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:03,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:03,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:05,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:06,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:34:06,994 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:34:09,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:11,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:12,433 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:12,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:13,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:15,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:17,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:18,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:18,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,215 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 01:34:19,417 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 01:34:19,488 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 01:34:19,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 01:34:19,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:19,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:21,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:22,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:34:22,221 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:34:22,921 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:34:23,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:24,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:24,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:25,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:27,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:28,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:28,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:29,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:31,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:32,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:33,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,412 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:34,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:35,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:37,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:34:37,453 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:34:37,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:39,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:40,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,128 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:40,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:41,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:42,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:42,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:43,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:43,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:44,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:45,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:47,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:48,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:48,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:49,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:49,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:50,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:51,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:52,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:34:52,759 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:34:53,001 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:34:53,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:34:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:34:56,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:03,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:08,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:35:08,082 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:35:09,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:12,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:13,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:16,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,418 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:16,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:17,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:18,430 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,431 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,449 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:18,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,805 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,811 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:19,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:21,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:23,058 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:35:23,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:35:23,384 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:35:23,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:23,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:25,739 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:29,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:29,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:30,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,758 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:32,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:33,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:35,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,664 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:35,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:37,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:37,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:35:38,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:35:38,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:38,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:39,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:41,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:45,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:45,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,804 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:45,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:47,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:49,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:50,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,061 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,062 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,450 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:50,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,468 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,469 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:51,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:53,126 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:35:53,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:53,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:35:53,873 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:35:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:55,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:57,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:35:58,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,209 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:35:58,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:01,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:02,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,610 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,617 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,667 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,668 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:02,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:03,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:05,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:05,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:07,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:09,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:36:09,136 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:36:09,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:11,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:14,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:14,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:15,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:17,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:17,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:18,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:20,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:22,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:23,191 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:36:24,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:36:24,413 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:36:24,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,194 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,195 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,199 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:26,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,730 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:28,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:30,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:32,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:34,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,352 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:34,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:36,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:36,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:37,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:37,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:38,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:39,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:36:39,693 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:36:40,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,510 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:40,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:41,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:42,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:43,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,615 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:43,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:44,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:44,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,420 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,422 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,424 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:45,429 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:46,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:48,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:50,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:50,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,502 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,505 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,506 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:51,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:36:52,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:36:53,257 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:36:54,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:36:54,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:37:02,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:02,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:02,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:03,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:06,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:10,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:37:10,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:37:12,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:16,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,040 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:18,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:20,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:22,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:23,329 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:37:25,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:25,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:37:25,434 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:37:26,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,919 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:26,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,175 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:28,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:31,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:33,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:34,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:34,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,508 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,515 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,516 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:37,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:39,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,167 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,171 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:39,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:40,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:37:40,940 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:37:43,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,258 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:43,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:44,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:44,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:45,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,421 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,425 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,426 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,428 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,432 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,434 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:47,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:51,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:51,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:53,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,121 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,123 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:53,396 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:37:53,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:56,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:37:56,173 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:37:57,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:37:57,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:57,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,136 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:37:59,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:01,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:01,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:03,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:05,807 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:06,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:06,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,155 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,156 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,157 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,161 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,994 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:07,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:09,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:11,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:38:11,401 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:38:11,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:12,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,122 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:12,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:13,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:14,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:14,801 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 01:38:14,964 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 01:38:15,031 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 01:38:15,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 01:38:15,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:16,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:16,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:17,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,165 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,166 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,168 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,170 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,172 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:17,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:19,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:21,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:23,481 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:38:23,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:25,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:26,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:26,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:38:26,715 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:38:27,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:29,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:29,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:31,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:31,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:33,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,371 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,373 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:35,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:37,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:37,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,993 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,995 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:37,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,483 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:38,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:39,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,986 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,987 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:40,990 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:41,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:41,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:38:41,955 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:38:43,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,033 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,034 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:43,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:44,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,127 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,130 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,131 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,132 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,134 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,137 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,142 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,143 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:45,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:47,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:47,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:47,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:38:49,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:38:53,547 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:38:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:38:57,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:38:59,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:05,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,413 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,416 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:05,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:09,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,452 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,453 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:09,991 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:09,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:10,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:11,564 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:12,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:39:12,490 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:39:13,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,660 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,662 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:17,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:21,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,288 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:21,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:23,620 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:39:24,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:24,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:25,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:27,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:39:27,858 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:39:27,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:29,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,151 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,153 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,160 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,162 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,163 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:29,164 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,245 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,247 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,251 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,252 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,253 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,257 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:31,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:35,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:37,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:38,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:38,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:39,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:40,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,169 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,174 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,210 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:40,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:41,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:39:43,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:39:43,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:43,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:45,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,927 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,931 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:45,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:48,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,699 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:48,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:49,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:51,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,514 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:51,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:52,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:52,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:53,694 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:39:53,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:55,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,709 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,710 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,712 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:57,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:39:58,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:39:58,530 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:39:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,403 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,404 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:39:59,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:00,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,315 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:00,316 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:01,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:03,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:05,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:09,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:10,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,833 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:10,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,568 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,577 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,581 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,582 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,583 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:11,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:12,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,711 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,715 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,716 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,721 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,722 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,723 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:12,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:13,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:13,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:13,674 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:13,675 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:13,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:40:13,766 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:40:13,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:14,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:14,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:15,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:17,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:19,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:21,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:22,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,561 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:22,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:23,766 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:40:23,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:25,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:27,990 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:28,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:28,992 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:40:28,992 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:40:29,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:30,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,233 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:30,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:31,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:33,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:34,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,493 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:34,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,912 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:35,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:36,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:36,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:37,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:39,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,864 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:40,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:41,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,436 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,440 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,441 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,445 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,447 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,448 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,566 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,567 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,569 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,570 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,571 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,572 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,573 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,574 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,576 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,578 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:41,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,908 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,909 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,911 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,916 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,918 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:42,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:43,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:43,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,254 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:40:44,254 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:40:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,936 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,937 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:44,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:45,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,648 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:45,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:46,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:48,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:50,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,802 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,806 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:50,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:40:51,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:53,837 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:40:59,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:40:59,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:40:59,585 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:41:03,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:05,503 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,504 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,512 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,513 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,519 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,524 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:05,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:08,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:09,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:11,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:11,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:13,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,369 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,370 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:13,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:14,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:41:14,907 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:41:15,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:19,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:20,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,653 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:20,658 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:22,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:23,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:23,910 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:41:27,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:27,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:27,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:29,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:30,133 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:41:30,133 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:41:33,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:34,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,357 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,620 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,621 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:34,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:35,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:37,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:41,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:41,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,343 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,351 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:41,354 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,945 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,946 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:42,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:43,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:45,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:45,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,314 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:45,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:41:45,359 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:41:46,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,451 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,461 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,462 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,464 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,465 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,602 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,609 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,612 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,613 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,614 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,684 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,685 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,690 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:46,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,910 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,913 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,914 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,915 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,917 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,920 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,921 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,922 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,924 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,925 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:47,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,266 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:48,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:49,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:51,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:51,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,580 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,584 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,586 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:51,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:53,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,041 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:53,979 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:41:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,135 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,139 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,141 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,144 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,145 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,146 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,147 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,148 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,149 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:54,150 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:55,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,649 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,651 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:55,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:41:59,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,092 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:41:59,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:42:00,654 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:42:00,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:00,988 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:01,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,377 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:02,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:03,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:06,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:06,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:06,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:08,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:09,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:09,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:09,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:09,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:10,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,314 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 01:42:10,518 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 01:42:10,586 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 01:42:10,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:10,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 01:42:11,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:11,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,057 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,058 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,059 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,060 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,063 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,064 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,067 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:11,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:12,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:14,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:14,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,738 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,739 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,740 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,741 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,744 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,745 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,746 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,747 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,748 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:14,749 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:15,876 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:42:15,877 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:42:16,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:17,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,532 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:17,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:18,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:19,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,212 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,218 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,219 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:19,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:20,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:22,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,756 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,759 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,760 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,762 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:23,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:24,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:24,060 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:42:26,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:26,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,866 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,874 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:26,875 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,518 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,520 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,522 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,523 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,527 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,529 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:27,530 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:28,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:30,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:31,126 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:42:31,126 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:42:32,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:34,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:34,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,091 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,099 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,102 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:34,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,466 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,467 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,470 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,471 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,472 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,473 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,474 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,476 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,477 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:35,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:36,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:37,375 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,376 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,378 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,379 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,380 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,381 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,383 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:37,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:38,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:40,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:40,355 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,356 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,358 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,359 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,362 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,363 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,364 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,365 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,366 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,367 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,368 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,798 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,799 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,800 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:40,965 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,790 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,791 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,792 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,793 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,794 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,795 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,796 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,797 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,877 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,881 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:41,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:42,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:43,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,271 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,273 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,274 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,279 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,305 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,306 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,308 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,309 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,310 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,311 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,312 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:43,313 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:44,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:46,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:46,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:42:46,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:42:54,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:42:54,121 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:42:56,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:56,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:42:58,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:01,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:43:01,633 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:43:04,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:07,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,235 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,246 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,249 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:07,250 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:08,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:10,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,808 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:10,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:12,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:14,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,521 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,525 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,526 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,528 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,533 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,535 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,537 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,868 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,870 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,872 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:14,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,073 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:16,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:17,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:43:17,035 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:43:18,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:21,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:21,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:22,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:24,059 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:24,193 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:43:27,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,623 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,624 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,625 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,632 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:27,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:28,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:30,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:30,587 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,588 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,591 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,593 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,594 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,601 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,603 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:30,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:32,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:43:32,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:43:33,226 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,227 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,234 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,243 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:33,244 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:34,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:36,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:40,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:42,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,173 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,177 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,178 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,179 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,187 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:43,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:44,326 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,327 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,337 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,487 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,494 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:44,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,850 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:46,985 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:47,657 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:43:47,658 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:43:48,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,626 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,627 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,629 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:49,638 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:50,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,405 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,411 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,547 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,552 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,557 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:50,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,763 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,767 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,768 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,773 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:51,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:53,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:54,264 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:43:55,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,666 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,672 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,673 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,676 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,677 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,678 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,679 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,680 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,851 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,859 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,860 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,861 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,862 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,863 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,900 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,903 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:56,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:57,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:43:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:58,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,765 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,766 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,769 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,770 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,772 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,775 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,776 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,779 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:43:59,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:01,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:02,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:44:02,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:44:03,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:03,335 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,336 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,339 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,340 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,341 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,342 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,344 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,345 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,346 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,348 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,349 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:03,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:05,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:06,196 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,197 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,198 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,605 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,606 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,607 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:06,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:07,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,585 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,589 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,590 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,592 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,595 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,596 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,598 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,599 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:08,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:09,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:10,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,272 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,275 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,276 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,278 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,280 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,281 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,282 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,284 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:10,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,553 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,554 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,556 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,558 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,559 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,560 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,562 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:11,563 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:13,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,075 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:15,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,083 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,085 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,086 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,087 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:15,088 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,002 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,013 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:16,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:17,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:18,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:44:18,110 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:44:19,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:20,255 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,259 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,260 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,261 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,262 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,263 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,265 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,267 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,268 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,269 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:20,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:21,947 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,948 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,951 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,952 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,953 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,959 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,960 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:21,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:23,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:24,330 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:44:25,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:26,180 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,181 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,183 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,184 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,186 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,188 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,189 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,190 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,191 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:26,193 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:27,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:29,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,989 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,996 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,997 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:29,999 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:30,000 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:30,001 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:31,777 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,778 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,780 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,781 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,782 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,783 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,785 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,787 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,788 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:31,789 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,385 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,388 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,389 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,392 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,395 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:32,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,023 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:33,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:33,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:44:33,394 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:44:35,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:36,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,822 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,823 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,824 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,826 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,827 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:36,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,809 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,812 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,813 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,815 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,816 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,817 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,818 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,819 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,820 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:37,821 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,117 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,119 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:38,120 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,003 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,009 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,010 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,011 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,012 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,016 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,017 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:39,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,974 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,975 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,976 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,977 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,978 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,979 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,980 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,981 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:39,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:41,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:43,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:48,622 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:44:48,622 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:44:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,035 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,037 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,039 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,047 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,048 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:53,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:54,400 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:44:54,949 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,954 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,955 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,956 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,957 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,966 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,970 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:54,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,437 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,438 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,442 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,444 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,446 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,455 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,456 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,457 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,458 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,459 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:55,460 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:57,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:44:59,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,843 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,844 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,852 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:44:59,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:01,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:03,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:45:03,847 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:45:05,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:08,888 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,889 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,893 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,896 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,901 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,902 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,904 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,905 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:08,907 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:09,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:11,089 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,090 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,094 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,095 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,096 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,097 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,098 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,103 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:11,109 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:13,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:14,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,228 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,229 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,230 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,237 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,238 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,239 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,240 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,241 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:14,242 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,065 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,068 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,069 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,071 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,076 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,078 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,079 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,080 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:16,082 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:17,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:19,077 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:45:19,078 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:45:19,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:21,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,038 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,043 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,044 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,045 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,046 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,049 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,050 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,051 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,052 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,053 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,054 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:21,055 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:23,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:24,478 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:45:25,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,286 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,289 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,290 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,291 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,292 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,299 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,301 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,302 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:25,303 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,475 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,478 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,480 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,481 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,482 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,486 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:28,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:29,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:31,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:32,841 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,842 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,845 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,846 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,847 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,848 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,853 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,854 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,855 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,856 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,857 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,858 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,878 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,879 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,883 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,884 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,885 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,886 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,890 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,891 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,892 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,894 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:32,895 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:34,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:45:34,329 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:45:35,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:37,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:39,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:39,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,387 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,390 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,391 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,396 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,399 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:39,402 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:43,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:46,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:48,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:48,318 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,319 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,322 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,323 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,324 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,325 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,328 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,329 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,331 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,332 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,333 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:48,334 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,485 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,488 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,489 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,490 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,491 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,492 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,495 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,496 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,497 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,498 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,499 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,501 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,630 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,631 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,634 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,635 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,636 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,637 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:45:49,701 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:45:49,713 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,718 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,719 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,720 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,724 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,725 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,726 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,727 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,728 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:49,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:50,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,536 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,539 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,540 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,541 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,542 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,543 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,546 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,548 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,549 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,551 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,639 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,640 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,644 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,647 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,650 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,652 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,654 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,655 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,686 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,687 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,688 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:50,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,205 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,206 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,207 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,208 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:52,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,015 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,019 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,020 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,021 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,022 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,024 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,025 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,026 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,027 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,028 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,029 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:53,030 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:54,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:54,555 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:45:56,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:56,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,694 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,695 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,696 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,697 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,698 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:56,705 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,692 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,693 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,700 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,701 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,702 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,703 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,704 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,707 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,708 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,928 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,929 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,932 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,933 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,934 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,935 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,938 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,940 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,941 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,942 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,943 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:57,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:58,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:45:59,100 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,101 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,104 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,105 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,106 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,108 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,110 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,111 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,113 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,114 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,116 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,825 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,828 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,829 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,830 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,831 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,832 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,834 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,835 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,836 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,837 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,838 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:45:59,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:00,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:01,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,962 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,963 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,967 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,968 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,971 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,972 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:01,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:02,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:03,982 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:03,983 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:03,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:03,984 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,211 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,214 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,215 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,216 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,217 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,220 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,221 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,222 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,223 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,225 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,401 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,407 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,408 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,409 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,410 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,579 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 01:46:04,674 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 01:46:04,886 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 01:46:04,954 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 01:46:05,005 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:46:05,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:46:05,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 01:46:06,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:08,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:10,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:12,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:14,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:16,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:18,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:20,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:20,347 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:46:20,349 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:46:22,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:24,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:24,969 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:46:26,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:28,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:30,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:32,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:34,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:35,642 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:46:35,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:46:36,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:38,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:40,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:42,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:44,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:46,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:48,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:50,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:50,871 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:46:50,871 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:46:52,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:54,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:55,041 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:46:56,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:46:58,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:00,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:02,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:04,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:06,138 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:47:06,138 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:47:06,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:08,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:10,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:12,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:14,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:16,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:18,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:20,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:21,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:47:21,439 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:47:22,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:24,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:25,116 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:47:27,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:29,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:31,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:33,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:35,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:36,681 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:47:36,681 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:47:37,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:39,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:41,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:43,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:45,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:47,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:49,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:51,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:52,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:47:52,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:47:53,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:55,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:55,186 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:47:57,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:47:59,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:01,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:03,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:05,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:07,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:07,320 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:48:07,320 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:48:09,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:11,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:13,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:15,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:17,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:19,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:21,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:22,550 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:48:22,550 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:48:23,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:25,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:25,262 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:48:27,188 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:29,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:31,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:33,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:35,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:37,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:37,810 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:48:37,810 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:48:39,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:41,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:43,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:45,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:47,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:49,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:51,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:53,036 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:48:53,036 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:48:53,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:55,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:55,337 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:48:57,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:48:59,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:01,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:03,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:05,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:07,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:08,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:49:08,288 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:49:09,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:11,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:13,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:15,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:17,210 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:19,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:21,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:23,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:23,619 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:49:23,620 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:49:25,214 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:25,413 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:49:28,215 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:30,216 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:32,217 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:34,218 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:36,219 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:38,220 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:39,007 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:49:39,008 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:49:40,221 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:42,222 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:44,223 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:46,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:48,224 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:50,225 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:52,226 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:54,227 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:54,231 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:49:54,231 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:49:55,489 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:49:56,228 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:49:58,229 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:00,230 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:02,231 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:04,232 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:06,233 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:08,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:09,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:50:09,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:50:10,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:12,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:14,236 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:16,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:18,238 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:20,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:22,239 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:24,240 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:24,803 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:50:24,804 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:50:25,562 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:50:26,241 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:28,242 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:30,243 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:32,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:34,244 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:36,245 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:38,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:40,032 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:50:40,032 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:50:40,247 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:42,248 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:44,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:46,249 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:48,250 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:50,251 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:52,252 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:54,253 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:55,293 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:50:55,294 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:50:55,634 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:50:57,254 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:50:59,255 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:01,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:03,256 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:05,257 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:07,258 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:09,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:10,689 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:51:10,690 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:51:11,259 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:13,260 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:15,261 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:17,262 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:19,263 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:21,264 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:23,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:25,265 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:25,708 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:51:25,958 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:51:25,958 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:51:27,266 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:29,267 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:31,268 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:33,269 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:35,270 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:37,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:39,271 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:41,264 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:51:41,264 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:51:41,272 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:43,273 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:45,274 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:47,275 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:49,276 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:51,277 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:53,278 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:55,279 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:55,783 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:51:56,618 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:51:56,618 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:51:57,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:51:59,280 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:01,281 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:04,282 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:06,283 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:08,284 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:10,285 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:11,840 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:52:11,840 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:52:12,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:14,286 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:16,287 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:18,288 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:20,289 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:22,290 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:24,291 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:25,854 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:52:26,292 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:27,066 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:52:27,066 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:52:28,293 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:30,294 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:32,296 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:34,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:36,297 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:38,298 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:40,299 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:42,300 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:42,350 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:52:42,351 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:52:44,301 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:46,302 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:48,303 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:50,304 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:52,305 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:54,306 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:55,927 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:52:56,307 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:52:57,774 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:52:57,775 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:52:58,308 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:00,309 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:02,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:04,310 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:06,311 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:08,312 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:10,313 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:12,314 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:13,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:53:13,019 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:53:14,315 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:16,316 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:18,317 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:20,318 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:22,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:24,319 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:26,004 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:53:26,320 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:28,321 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:28,360 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:53:28,360 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:53:30,322 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:32,323 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:34,324 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:36,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:38,325 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:40,326 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:42,327 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:43,597 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:53:43,598 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:53:44,328 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:47,329 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:49,330 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:51,331 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:53,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:55,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:56,079 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:53:57,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:53:58,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:53:58,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:53:59,334 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:01,335 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:03,336 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:05,337 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:07,338 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:09,339 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:11,340 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:13,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:14,185 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:54:14,185 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:54:15,341 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:17,342 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:19,343 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:21,344 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:23,345 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:25,346 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:26,155 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:54:27,347 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:29,348 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:29,415 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:54:29,415 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:54:31,349 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:33,350 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:35,351 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:37,352 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:39,353 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:41,354 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:43,355 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:44,633 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:54:44,634 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:54:45,356 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:47,357 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:49,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:51,358 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:53,359 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:55,360 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:56,227 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:54:57,361 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:54:59,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:00,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:55:00,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:55:01,362 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:03,363 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:05,364 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:07,365 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:09,366 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:11,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:13,367 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:15,256 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:55:15,256 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:55:15,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:17,368 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:19,369 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:21,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:23,370 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:25,371 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:26,301 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:55:27,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:29,372 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:30,484 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:55:30,484 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:55:31,373 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:33,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:35,374 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:38,375 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:40,376 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:42,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:44,377 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:45,706 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:55:45,706 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:55:46,378 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:48,379 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:50,380 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:52,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:54,381 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:56,375 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:55:56,382 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:55:58,383 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:00,384 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:00,969 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:56:00,970 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:56:02,385 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:04,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:06,386 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:08,387 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:10,388 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:12,389 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:14,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:16,384 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:56:16,385 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:56:16,390 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:18,391 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:20,392 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:22,393 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:24,394 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:26,395 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:26,451 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:56:28,396 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:30,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:31,729 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:56:31,729 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:56:32,397 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:34,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:36,399 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:38,400 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:40,401 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:42,402 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:44,403 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:46,404 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:46,998 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:56:46,999 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:56:48,405 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:50,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:52,406 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:54,407 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:56,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:56:56,534 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:56:58,408 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:00,409 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:02,307 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:57:02,308 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:57:02,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:04,410 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:06,411 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:08,412 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:10,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:12,413 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:14,414 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:16,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:17,669 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:57:17,669 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:57:18,415 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:20,416 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:22,417 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:25,418 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:26,608 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:57:27,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:29,419 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:31,420 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:32,897 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:57:32,897 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:57:33,421 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:35,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:37,422 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:39,423 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:41,424 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:43,425 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:45,426 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:47,427 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:48,213 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:57:48,213 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:57:49,428 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:51,429 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:53,430 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:55,431 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:56,684 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:57:57,432 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:57:59,433 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:01,434 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:03,435 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:03,439 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:58:03,440 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:58:05,436 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:07,437 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:09,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:11,438 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:13,439 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:15,440 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:17,441 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:18,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:58:18,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:58:19,442 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:21,443 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:24,444 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:26,445 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:26,761 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:58:28,446 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:30,447 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:32,448 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:33,961 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:58:33,961 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:58:34,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:36,449 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:38,450 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:40,451 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:42,452 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:44,453 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:46,454 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:48,455 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:49,200 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:58:49,200 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:58:50,456 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:52,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:54,457 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:56,458 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:58:56,839 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:58:58,459 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:00,460 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:02,461 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:04,462 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:04,517 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:59:04,518 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:59:06,463 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:08,464 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:10,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:12,465 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:14,466 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:16,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:18,467 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:19,755 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:59:19,756 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:59:20,468 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:22,469 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:24,470 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:26,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:26,913 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:59:28,472 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:30,473 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:32,474 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:34,475 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:35,004 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:59:35,005 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:59:36,476 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:38,477 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:40,478 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:42,479 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:44,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:46,480 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:48,481 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:50,298 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 01:59:50,299 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 01:59:50,482 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:52,483 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:54,484 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:56,987 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 01:59:57,485 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 01:59:59,486 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:01,487 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:03,488 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:05,489 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:05,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:00:05,531 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:00:07,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:09,490 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:11,491 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:13,492 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:15,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:17,493 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:19,494 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:20,839 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:00:20,839 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:00:21,495 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:23,496 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:25,497 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:27,059 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:00:27,498 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:29,499 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:31,500 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:33,501 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:35,502 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:36,285 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:00:36,285 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:00:37,503 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:39,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:41,504 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:43,505 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:45,506 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:47,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:49,507 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:51,508 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:51,608 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:00:51,608 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:00:53,509 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:55,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:57,132 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:00:57,510 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:00:59,511 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:01,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:03,512 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:05,513 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:06,873 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:01:06,874 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:01:07,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:09,514 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:11,515 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:13,516 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:15,517 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:17,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:19,518 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:21,519 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:22,112 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:01:22,112 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:01:23,520 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:25,521 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:27,207 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:01:27,522 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:29,523 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:31,524 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:34,525 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:36,526 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:37,330 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:01:37,330 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:01:38,527 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:40,528 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:42,529 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:44,530 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:46,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:48,531 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:50,532 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:52,533 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:52,646 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:01:52,646 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:01:54,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:56,534 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:01:57,260 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:01:58,535 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:00,536 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:02,537 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:04,538 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:06,539 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:07,926 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:02:07,926 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:02:08,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:10,540 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:12,541 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:14,542 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:16,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:18,543 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:20,544 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:22,545 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:23,182 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:02:23,182 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:02:24,546 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:26,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:27,332 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:02:28,547 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:30,548 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:32,549 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:34,550 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:36,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:38,406 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:02:38,407 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:02:38,551 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:40,552 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:42,553 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:44,554 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:46,555 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:48,556 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:50,557 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:52,558 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:53,733 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:02:53,733 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:02:54,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:56,559 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:02:57,405 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:02:58,560 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:00,561 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:02,562 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:04,563 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:06,564 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:08,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:09,031 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:03:09,031 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:03:10,565 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:12,566 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:14,567 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:16,568 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:19,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:21,570 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:23,571 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:24,338 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:03:24,338 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:03:25,572 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:27,481 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:03:27,573 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:29,574 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:31,575 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:33,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:35,576 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:37,577 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:39,578 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:39,641 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:03:39,642 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:03:41,579 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:43,580 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:45,581 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:47,582 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:49,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:51,583 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:53,584 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:54,899 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:03:54,899 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:03:55,585 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:57,556 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:03:57,586 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:03:59,587 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:01,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:03,588 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:05,589 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:07,590 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:09,591 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:10,125 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:04:10,125 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:04:11,592 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:13,593 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:15,594 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:17,595 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:19,596 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:21,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:23,597 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:25,509 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:04:25,510 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:04:26,599 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:27,630 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:04:28,600 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:30,601 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:32,602 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:34,603 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:36,604 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:38,605 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:40,606 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:40,764 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:04:40,764 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:04:42,607 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:44,608 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:46,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:48,609 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:50,610 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:52,611 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:54,612 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:56,070 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:04:56,071 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:04:56,613 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:04:57,705 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:04:58,614 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:00,615 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:02,616 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:04,617 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:06,618 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:08,619 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:10,620 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:11,297 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:05:11,297 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:05:12,621 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:14,622 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:16,623 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:19,624 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:21,625 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:23,626 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:25,627 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:26,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:05:26,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:05:27,628 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:27,780 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:05:29,629 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:31,630 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:33,631 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:35,632 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:37,633 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:39,634 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:41,635 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:41,880 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:05:41,880 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:05:43,636 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:45,637 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:47,638 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:49,639 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:51,640 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:53,641 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:55,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:57,176 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:05:57,177 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:05:57,642 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:05:57,854 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:05:59,643 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:01,644 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:03,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:05,645 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:07,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:09,646 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:11,647 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:12,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:06:12,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:06:13,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:15,648 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:17,649 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:19,650 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:21,651 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:23,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:25,652 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:27,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:06:27,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:06:27,653 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:27,930 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:06:29,654 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:31,655 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:33,656 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:35,657 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:37,658 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:40,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:42,660 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:42,865 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:06:42,865 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:06:44,661 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:46,662 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:48,663 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:50,664 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:52,665 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:54,666 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:56,667 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:06:58,007 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:06:58,201 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:06:58,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:06:58,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:00,668 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:02,670 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:04,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:06,671 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:08,672 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:10,673 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:12,674 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:13,427 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:07:13,428 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:07:14,675 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:16,676 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:18,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:20,677 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:22,678 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:24,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:26,679 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:28,083 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:07:28,670 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:07:28,670 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:07:28,680 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:30,681 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:32,682 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:34,683 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:36,684 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:38,685 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:40,686 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:42,687 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:43,906 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:07:43,906 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:07:44,688 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:46,689 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:48,690 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:50,691 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:52,692 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:54,693 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:56,694 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:58,158 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:07:58,695 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:07:59,158 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:07:59,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:08:00,696 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:02,697 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:04,698 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:07,699 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:09,700 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:11,701 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:13,702 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:14,400 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:08:14,400 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:08:15,703 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:17,704 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:19,705 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:21,706 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:23,707 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:25,708 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:27,709 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:28,236 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:08:29,643 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:08:29,643 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:08:29,710 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:31,711 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:33,712 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:35,713 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:37,714 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:39,715 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:41,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:43,716 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:44,950 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:08:44,950 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:08:45,717 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:47,718 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:49,719 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:51,720 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:53,721 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:55,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:57,722 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:08:58,310 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:08:59,723 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:00,192 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:09:00,192 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:09:01,724 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:03,725 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:05,726 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:07,727 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:09,728 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:11,729 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:13,730 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:15,511 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:09:15,512 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:09:15,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:17,731 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:19,732 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:21,733 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:23,734 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:25,735 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:27,736 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:28,388 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:09:29,737 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:30,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:09:30,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:09:31,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:33,740 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:35,741 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:37,742 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:39,743 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:42,744 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:44,745 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:45,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:09:45,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:09:46,746 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:48,747 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:50,748 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:52,749 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:54,750 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:56,751 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:09:58,464 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:09:58,752 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:00,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:01,287 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:10:01,287 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:10:02,753 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:04,754 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:06,755 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:08,756 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:10,757 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:12,758 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:14,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:16,534 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:10:16,534 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:10:16,759 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:18,760 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:20,761 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:22,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:24,762 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:26,763 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:28,540 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:10:28,764 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:30,765 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:31,761 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:10:31,761 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:10:32,766 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:34,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:36,767 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:38,768 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:40,769 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:42,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:44,770 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:46,771 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:47,072 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:10:47,072 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:10:48,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:50,772 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:52,773 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:54,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:56,774 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:10:58,617 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:10:58,775 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:00,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:02,443 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:11:02,443 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:11:02,776 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:05,777 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:07,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:09,778 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:11,779 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:13,780 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:15,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:17,671 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:11:17,671 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:11:17,781 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:19,782 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:21,783 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:23,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:25,784 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:27,785 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:28,691 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:11:29,786 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:31,787 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:33,018 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:11:33,018 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:11:33,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:35,788 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:37,789 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:39,790 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:41,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:43,792 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:45,793 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:47,794 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:48,393 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:11:48,393 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:11:49,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:51,795 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:53,796 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:55,797 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:57,798 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:11:58,770 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:11:59,799 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:01,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:03,661 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:12:03,662 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:12:03,800 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:05,801 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:07,802 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:09,803 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:11,804 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:13,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:15,805 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:17,806 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:18,887 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:12:18,888 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:12:20,808 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:22,809 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:24,810 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:26,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:28,811 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:28,843 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:12:30,812 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:32,813 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:34,154 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:12:34,155 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:12:34,814 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:36,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:38,815 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:40,816 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:42,817 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:44,818 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:46,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:48,819 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:49,382 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:12:49,383 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:12:50,820 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:52,821 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:54,822 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:56,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:58,823 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:12:58,917 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:13:00,824 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:02,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:04,682 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:13:04,683 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:13:04,825 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:06,826 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:08,827 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:10,828 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:13,829 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:15,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:17,830 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:19,831 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:19,944 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:13:19,944 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:13:21,832 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:23,833 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:25,834 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:27,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:28,991 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:13:29,835 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:31,836 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:33,837 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:35,232 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:13:35,232 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:13:35,838 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:37,839 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:39,840 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:41,841 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:43,842 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:45,843 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:47,844 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:49,845 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:50,454 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:13:50,455 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:13:51,846 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:53,847 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:55,848 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:57,849 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:13:59,065 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:13:59,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:01,850 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:03,851 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:05,751 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:14:05,751 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:14:05,852 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:07,853 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:09,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:11,854 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:13,855 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:15,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:17,856 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:19,857 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:21,115 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:14:21,115 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:14:21,858 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:23,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:25,859 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:27,860 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:29,142 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:14:29,861 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:31,862 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:33,863 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:35,868 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:36,361 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:14:36,362 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:14:37,869 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:39,870 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:42,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:44,872 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:46,873 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:48,874 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:50,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:51,656 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:14:51,656 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:14:52,875 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:54,876 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:56,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:58,877 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:14:59,216 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:15:00,878 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:02,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:04,879 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:06,880 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:06,882 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:15:06,882 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:15:08,881 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:10,882 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:12,883 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:14,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:16,884 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:18,885 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:20,886 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:22,202 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:15:22,202 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:15:22,887 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:24,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:26,888 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:28,889 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:29,289 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:15:30,890 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:32,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:34,891 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:36,892 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:37,575 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:15:37,575 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:15:38,893 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:40,894 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:42,895 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:44,896 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:46,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:48,897 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:50,898 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:52,814 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:15:52,814 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:15:52,899 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:54,900 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:56,901 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:58,902 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:15:59,367 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:16:00,903 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:02,904 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:04,905 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:06,906 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:08,084 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:16:08,084 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:16:08,907 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:10,908 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:12,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:14,909 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:16,910 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:18,911 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:20,912 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:23,419 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:16:23,419 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:16:23,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:25,913 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:27,914 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:29,443 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:16:29,915 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:31,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:33,916 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:35,917 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:37,918 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:38,659 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:16:38,660 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:16:39,919 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:41,920 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:43,921 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:45,922 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:47,923 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:49,924 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:51,925 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:53,926 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:53,964 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:16:53,964 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:16:55,927 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:57,928 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:16:59,518 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:16:59,929 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:01,930 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:03,931 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:05,932 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:07,933 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:09,270 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:17:09,271 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:17:09,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:11,934 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:13,935 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:15,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:17,936 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:19,937 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:21,938 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:23,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:24,500 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:17:24,501 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:17:25,939 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:27,940 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:29,598 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:17:29,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:31,941 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:33,942 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:35,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:37,943 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:39,735 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:17:39,736 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:17:40,944 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:42,945 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:44,946 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:46,947 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:48,948 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:50,949 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:52,950 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:54,951 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:55,081 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:17:55,081 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:17:56,952 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:58,953 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:17:59,672 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:18:00,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:02,954 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:04,955 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:06,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:08,956 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:10,353 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:18:10,353 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:18:10,957 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:12,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:14,958 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:16,959 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:18,960 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:20,961 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:22,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:24,962 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:25,611 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:18:25,611 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:18:26,963 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:28,964 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:29,748 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:18:30,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:32,965 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:34,966 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:36,967 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:38,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:40,939 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:18:40,939 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:18:40,968 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:42,969 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:44,970 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:46,971 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:48,972 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:50,973 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:52,974 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:54,975 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:56,304 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:18:56,305 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:18:56,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:58,976 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:18:59,825 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:19:00,977 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:02,978 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:04,979 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:06,980 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:08,981 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:11,545 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:19:11,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:19:11,982 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:13,983 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:15,984 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:17,985 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:19,986 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:21,987 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:23,988 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:25,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:26,849 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:19:26,849 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:19:27,989 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:29,901 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:19:29,991 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:31,992 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:33,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:35,993 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:37,994 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:39,995 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:41,996 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:42,107 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:19:42,108 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:19:43,997 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:45,998 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:47,999 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:50,000 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:52,001 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:54,002 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:56,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:57,397 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:19:57,397 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:19:58,003 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:19:59,976 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:20:00,004 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:02,005 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:04,006 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:06,007 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:08,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:10,008 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:12,009 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:12,663 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:20:12,663 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:20:14,010 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:16,011 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:18,012 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:20,013 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:22,014 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:24,015 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:26,016 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:27,930 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:20:27,930 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:20:28,017 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:30,018 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:30,052 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:20:32,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:34,019 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:36,020 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:38,021 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:40,022 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:42,023 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:43,224 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:20:43,224 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:20:44,024 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:46,025 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:48,026 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:51,027 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:53,028 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:55,029 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:57,030 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:20:58,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:20:58,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:20:59,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:00,131 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:21:01,031 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:03,032 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:05,033 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:07,034 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:09,035 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:11,036 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:13,037 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:13,784 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:21:13,784 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:21:15,038 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:17,039 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:19,040 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:21,041 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:23,042 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:25,043 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:27,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:29,044 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:29,159 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:21:29,159 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:21:30,208 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:21:31,045 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:33,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:35,046 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:37,047 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:39,048 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:41,049 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:43,050 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:44,398 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:21:44,398 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:21:45,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:47,051 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:49,052 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:51,053 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:53,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:55,054 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:57,055 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:59,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:21:59,683 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:21:59,683 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:22:00,282 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:22:01,056 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:03,057 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:05,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:07,058 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:10,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:12,060 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:14,061 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:14,923 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:22:14,924 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:22:16,062 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:18,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:20,063 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:22,064 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:24,065 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:26,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:28,066 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:30,067 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:30,152 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:22:30,152 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:22:30,381 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:22:32,068 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:34,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:36,069 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:38,070 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:40,071 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:42,072 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:44,073 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:45,417 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:22:45,417 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:22:46,074 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:48,075 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:50,076 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:52,077 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:54,078 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:56,079 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:22:58,080 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:00,081 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:00,436 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:23:00,645 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:23:00,645 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:23:02,082 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:04,083 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:06,084 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:08,085 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:10,086 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:12,087 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:14,088 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:15,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:23:15,869 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:23:16,089 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:18,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:20,090 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:22,091 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:24,092 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:26,093 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:28,094 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:30,095 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:30,512 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:23:31,236 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:23:31,237 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:23:32,096 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:34,097 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:36,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:38,098 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:40,099 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:42,100 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:44,101 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:46,102 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:46,531 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:23:46,532 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:23:48,103 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:51,104 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:53,105 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:55,106 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:57,107 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:23:59,108 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:00,586 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:24:01,109 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:01,807 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:24:01,808 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:24:03,110 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:05,111 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:07,112 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:09,113 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:11,114 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:13,115 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:15,116 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:17,117 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:17,118 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:24:17,118 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:24:19,118 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:21,119 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:23,120 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:25,121 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:27,122 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:29,123 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:30,662 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:24:31,124 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:32,374 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:24:32,374 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:24:33,125 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:35,126 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:37,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:39,127 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:41,128 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:43,129 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:45,130 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:47,131 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:47,600 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:24:47,601 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:24:49,132 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:51,133 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:53,134 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:55,135 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:57,136 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:24:59,137 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:00,736 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:25:01,138 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:03,008 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:25:03,009 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:25:04,139 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:06,140 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:08,141 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:10,142 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:12,143 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:14,144 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:16,145 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:18,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:18,277 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:25:18,277 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:25:20,146 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:22,147 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:24,148 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:26,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:28,149 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:30,150 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:30,815 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:25:32,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:33,507 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:25:33,507 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:25:34,151 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:36,152 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:38,153 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:40,154 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:42,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:44,155 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:46,156 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:48,157 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:48,736 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:25:48,737 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:25:50,158 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:52,159 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:54,160 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:56,161 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:25:58,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:00,162 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:00,892 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:26:02,163 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:04,042 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:26:04,043 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:26:04,164 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:06,165 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:08,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:10,166 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:12,167 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:14,168 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:16,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:18,169 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:19,317 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:26:19,318 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:26:20,170 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:22,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:24,171 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:26,172 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:28,173 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:30,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:30,973 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:26:32,174 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:34,175 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:34,544 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:26:34,545 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:26:36,176 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:39,177 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:41,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:43,178 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:45,179 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:47,180 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:49,181 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:49,771 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:26:49,772 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:26:51,182 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:53,183 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:55,184 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:57,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:26:59,185 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:01,048 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:27:01,186 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:03,187 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:05,093 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:27:05,093 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:27:05,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:07,189 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:09,190 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:11,191 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:13,192 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:15,193 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:17,194 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:19,195 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:20,386 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:27:20,386 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:27:21,196 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:23,197 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:25,198 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:27,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:29,199 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:31,125 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:27:31,200 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:33,201 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:35,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:35,717 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:27:35,717 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:27:37,202 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:39,203 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:41,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:43,204 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:45,205 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:47,206 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:49,207 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:51,056 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:27:51,056 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:27:51,208 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:53,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:55,209 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:27:58,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:28:00,211 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:28:01,198 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:28:02,212 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:28:04,213 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:28:06,283 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:28:06,283 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:28:21,538 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:28:21,538 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:28:31,274 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:28:36,869 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:28:36,870 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:28:52,140 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:28:52,140 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:29:01,347 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:29:01,691 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: partial_history 2022-06-24 02:29:01,697 DEBUG SenderThread:384207 [sender.py:send():232] send: history 2022-06-24 02:29:01,748 DEBUG SenderThread:384207 [sender.py:send():232] send: summary 2022-06-24 02:29:01,820 INFO SenderThread:384207 [sender.py:_save_file():946] saving file wandb-summary.json with policy end 2022-06-24 02:29:02,234 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/wandb-summary.json 2022-06-24 02:29:04,235 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:29:07,394 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:29:07,395 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:29:08,237 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:29:22,628 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:29:22,629 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:29:28,246 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:29:31,429 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:29:37,867 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:29:37,868 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:29:53,124 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:29:53,124 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:30:01,516 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:30:08,372 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:30:08,372 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:30:23,604 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:30:23,604 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:30:31,602 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:30:39,006 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:30:39,007 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:30:54,321 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:30:54,322 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:31:01,687 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:31:09,555 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:31:09,556 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:31:24,786 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:31:24,787 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:31:31,766 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:31:40,014 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:31:40,015 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:31:55,248 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:31:55,249 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:32:01,850 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:32:10,479 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:32:10,480 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:32:25,332 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:32:25,714 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:32:25,714 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:32:26,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:32:27,333 INFO Thread-7 :384207 [dir_watcher.py:_on_file_modified():226] file/dir modified: /mnt/lv_ai_1_dante/ml/models/wav2vec2-1b-npsc-nst-bokmaal/wandb/run-20220622_135628-3qxrqni1/files/output.log 2022-06-24 02:32:31,934 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:32:40,973 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:32:40,973 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:32:56,204 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:32:56,205 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status 2022-06-24 02:33:02,018 DEBUG SenderThread:384207 [sender.py:send():232] send: stats 2022-06-24 02:33:11,463 DEBUG HandlerThread:384207 [handler.py:handle_request():141] handle_request: stop_status 2022-06-24 02:33:11,464 DEBUG SenderThread:384207 [sender.py:send_request():246] send_request: stop_status